...
Submitting inference jobs to the LLM service is accomplished through the jobs API endpoint:
|
...
| Code Block | ||
|---|---|---|
| ||
app // Name of the application; in this case, "charityenginegridrepublic:text-inference" commandLine // JSON string of the prompt in an array named "input" tag // Name of the model to use for inference |
...
| Code Block | ||
|---|---|---|
| ||
{
"app": "charityenginegridrepublic:text-inference",
"commandLine": "{'input': ['When does 1+1=10?'] }",
"tag": "gemma:2b"
} |
...
The WebUI and notebook application both use this API.
TODO:
...
- Specify the response format from GET jobs/{id}
...