...
This process involves branching from a base model and training the branch on specific domain data in order to establish an expert, which routing logic is then able to activate in serving inference requests. The result is a framework for domain expertise that is easily-extensible, modular, and efficient.
Contents
| Table of Contents | ||||
|---|---|---|---|---|
|
Prerequisites
M*DEL's Aurora model (aurora-m) will be used as the base and RunPod compute resources will be used for training. Also, Weights and Biases will collect information during training for monitoring status, evaluating progress, and to allow comparing subsequent training runs for performance.
...
| Note |
|---|
If the following error appears when launching axolotl.cli.train: Error while finding module specification for 'axolotl.cli.train' (ModuleNotFoundError: No module named 'axolotl') |
...
This command will show a link with a gradio.live URL, which provides an interface to the model for testing inference. This is likely not sufficient for chatbook integration, but is robust enough to demonstrate functionality of the new expert.
Note: If you want to inference using an alternative base model you can run the command without specifying a lora_model_dir. This will work for any base model. Aurora and Starcoder examples below:
| Code Block |
|---|
$ accelerate launch -m axolotl.cli.inference examples/aurora/experiment1.yml --gradio # this will inference aurora
$ accelerate launch -m axolotl.cli.inference examples/starcoder/lora.yml --gradio # this will inference starcoder |
Upload the Expert to HuggingFace
...