Guide
Saving
Saving a trained model: the adapter alone, or the base model with the adapter merged in.
After a training job, result.save("out") saves the adapter, and result.save("out", merged=True) saves a full 16-bit model that any Transformers or vLLM server loads, from LoRA and QLoRA alike. With the lower-level calls, the model has save_pretrained_merged and push_to_hub_merged, which also uploads to a Hugging Face Hub repository.
After a training job
result = job.train()
result.save("qwen-adapter") # the adapter alone
result.save("qwen-tuned", merged=True) # the base model with the adapter merged in, in 16 bits
meepsquared train job.yaml saves the adapter in the job’s output folder when the training ends.
result.save writes to a folder. To upload to the Hugging Face Hub, use the model of the job’s trainer, which has the lower-level calls below:
trainer = job.trainer() # the trainer job.train() ran, with the trained model
trainer.model.push_to_hub_merged("my-org/qwen-tuned", trainer.processing_class, save_method="merged_16bit",
token=hf_token)
Save methods
| Method | Writes |
|---|---|
The adapter: |
The adapter alone, as PEFT saves it. Small; it needs the base model to run. |
Merged: |
The base model with |
With the lower-level calls
A model from get_peft_model has two methods:
model.save_pretrained_merged("qwen-tuned", tokenizer, save_method="merged_16bit")
model.push_to_hub_merged("my-org/qwen-tuned", tokenizer, save_method="merged_16bit", token=hf_token)
push_to_hub_merged creates the repository if it does not exist (private=True makes it private) and returns its URL.
The model in memory does not change: training or generation can go on after a save.
Several processes
In a run started with torchrun, every process calls save_pretrained_merged or push_to_hub_merged, and the first one writes or uploads; the others return at once (push_to_hub_merged returns "" in them). When the model is sharded across the processes with FSDP2, the adapter’s weights are gathered from all of them first, which is why every process has to make the call: one process alone would wait for the others forever. A merged save is refused in that case; see FSDP2.
Limits
-
A 4-bit merged save would lose precision and is refused; save 16-bit, then quantize with the tool of your serving backend.
-
GGUF export is not offered; convert a 16-bit save with llama.cpp’s converter.
-
A merged save takes one active adapter, and refuses DoRA adapters and a model sharded across processes.