...

--header 'Authorization: Bearer example-token'

Endpoints

...

Jobs on the LLM service can be submitted and managed through API requests.

Submitting a job

Submitting inference jobs to the LLM service is accomplished through the jobs API endpoint:

...

app // Name of the application; in this case, "gridrepublic:text-inference"
commandLine // JSON string of the prompt in an array named "inputs"; double-quotes must be escaped
hours // Runtime limit for the job; by default, use: 1
tag // Name of the model to use for inference

...

{
  "app": "gridrepublic:text-inference",
  "commandLine": "{\"inputs\": [\"When does 1+1=10?\"] }",
  "hours": 1,
  "tag": "gemma:2b"
}

When the job is submitted, the API returns a success indicator and either an array of job "ids", when success is true, or a string indicating an "error", when success is false. For example:

...

{
  "success":false,
  "error":"Invalid input format"
}

...

Retrieving job status

To retrieve the current status of a job that has been submitted to the LLM service, the jobs API endpoint accepts GET requests with a comma-separated list of one or more job IDs as a path parameter:

...