...
This command will show a link with a gradio.live URL, which provides an interface to the model for testing inference. This is likely not sufficient for chatbook integration, but is robust enough to demonstrate functionality of the new expert.
If you want to inference the base model you can run the command without specifying a lora_model_dir. This will work for any base model. Aurora and Starcoder examples below:
| Code Block |
|---|
$ accelerate launch -m axolotl.cli.inference examples/aurora/experiment1.yml --gradio # this will inference aurora
$ accelerate launch -m axolotl.cli.inference examples/starcoder/lora.yml --gradio # this will inference starcoder |
Upload the Expert to HuggingFace
...