Page History

If you want to inference the base model you can run the command without specifying a lora_model_dir. This will work for any base model. Aurora and Starcoder examples below:

$ accelerate launch -m axolotl.cli.inference examples/aurora/experiment1.yml --gradio # this will inference aurora 

$ accelerate launch -m axolotl.cli.inference examples/starcoder/lora.yml --gradio # this will inference starcoder