...
The core server, available as a docker Docker application, provides a local interactive inference service, via simple command line interface. A range of models are supported.
System requirements
Docker Engine must be installed and running on the target system and network connectivity is needed so that the Docker image and any specified models can be downloaded automatically.
The application container itself requires 600 MB of disk space and will use 400 MB of system memory to run. In addition to this, the model selected will have its own resource requirements. See the instructions on Choosing a model for further details.
 Installation and basic usage
...
| Code Block | ||
|---|---|---|
| ||
>>> |
Choosing a model
The application container requires 600 MB of disk space and will use 400 MB of system memory to run. In addition to this, the Zephyr 7B model that is used by default requires a bit over 4 GB of disk space and at least 4.5 GB of RAM to run. Depending on the resources available, smaller, lighter weight models and larger, more performant models can also be used.
...