...

The core server, available as a Docker application, provides a local interactive inference service, via simple command line interface. A range of models are supported.

System requirements

• Docker Engine must be installed and running on the target system

...

• Network connectivity is needed so that the Docker image and any specified models can be downloaded automatically

...

• Disk space and RAM:
  • The application container itself requires 600 MB of disk space and will use 400 MB of system memory to run.
  • In addition to

...

• • the above, the model selected will have its own resource requirements. See the instructions on Choosing a model for further details.

 Installation and basic usage

...

The Zephyr 7B model that is used by default requires a bit over 4 GB of disk space and at least 4.5 GB of RAM to run (in addition to the application container requirements, as outlined above). Depending on the resources available, smaller, lighter weight models and larger, more performant models can also be used.

...