Reference
API
Every public name of Meep² and its arguments.
import meepsquared as m2. m2.TrainingJob describes a training job; m2.Recipe and m2.Overrides change Meep²'s choices; job.plan() returns an m2.Plan and job.train() an m2.Result. Under them are the lower-level calls: m2.MeepLanguageModel and m2.MeepModel load a model and add LoRA; m2.fsdp_config gives the FSDP2 settings; meepsquared.chat_templates.train_on_responses_only puts the loss on the answers. Importing Meep² also adjusts TRL’s trainers; see Trainer settings.
TrainingJob
m2.TrainingJob(model, data=None, task="sft", method="lora", steps=None, epochs=None, rewards=None, output=None, split="train", report_to=None, eval_data=None, eval_split=None, eval_share=None, eval_every=None, timed_steps=None, tokenizer=None, recipe=Recipe(), overrides=Overrides()). Training jobs explains each field and each default, with its source.
| Field | Default | Meaning |
|---|---|---|
|
required |
A Hugging Face model id, a local folder, or a model loaded with |
|
required |
A Hugging Face dataset id, a local |
|
|
|
|
|
|
|
one epoch |
How long. Give one, not both. |
|
required for GRPO |
Reward functions as TRL takes them, or their names as |
|
|
The folder TRL writes checkpoints to. |
|
|
The split of a Hugging Face dataset. |
|
|
Where the trainer reports each step’s metrics, as Transformers' |
|
|
SFT only: evaluation data, as |
|
|
The split of |
|
|
SFT only: the share of the training data held out for evaluation, above 0 and at most 0.5, chosen with the trainer’s seed before the micro-batches are laid out, never trained on. |
|
about 10 evaluations over the planned steps |
Optimizer steps between evaluations (and one at the last step). Needs |
|
|
SFT only: the optimizer steps the plan times, after one to warm up, for its time per step; |
|
|
The tokenizer, when |
|
|
The training choices; see Recipe. |
|
|
The hardware choices; see Overrides. |
| Method | Does |
|---|---|
|
Loads the model, adds LoRA, measures the memory and builds TRL’s trainer; returns the Plan. Nothing is trained. A second call returns the same plan. |
|
Prints the plan (unless |
|
TRL’s |
|
The job as plain data. A loaded model, a |
|
The same as YAML. |
The command meepsquared train job.yaml (also python -m meepsquared train job.yaml) plans, trains and saves the adapter in the job’s output folder.
| Option | Does |
|---|---|
|
Prints the plan and stops; nothing is trained. Not with the options below. |
|
Prints the plan and the result as JSON. |
|
Writes the training’s state as JSON, whole, at the start, after the plan, after every optimizer step, at each checkpoint and at the end: |
|
When this file exists at the end of an optimizer step, the training saves a checkpoint in the output folder and stops; the state is |
|
Continues from the latest checkpoint in the output folder: the optimizer, the scheduler and the next micro-batch. Refused when there is no checkpoint, naming the folder. |
A SIGTERM stops the training as the stop file does. The exit code is 0 when the training completed (the adapter is saved), 75 when it stopped when asked (a checkpoint is saved, and no adapter), and any other when it failed (the status file’s error says why). See Stopping and resuming.
What a training job does not do: FSDP2 and a model spread over several GPUs (under torchrun a job trains with DDP; see Several GPUs), and GRPOConfig settings outside the recipe, such as beta. For these, use the lower-level calls below.
Recipe
m2.Recipe(…): the training choices. A field left as None is chosen by auto mode, and the plan says how.
| Field | Auto mode | Meaning |
|---|---|---|
|
|
LoRA’s rank. |
|
the rank |
LoRA’s alpha; the update is scaled by alpha / rank. |
|
|
LoRA’s dropout. |
|
the seven projections |
The module names LoRA adapts: |
|
|
The peak learning rate. Taken as given, not scaled. |
|
|
|
|
100, or 5% of the planned steps when fewer |
The steps of the warm-up. |
|
|
Examples per update of the weights; for GRPO, completions per update. |
|
off |
SFT only: the batch as tokens per update instead; each update takes as many conversations as fit. |
|
|
A Transformers optimizer name. |
|
|
Tokens per example; for GRPO, the prompt and the completion. |
|
|
The largest gradient norm; a larger gradient is scaled down to it. |
|
|
The optimizer’s weight decay. |
|
|
|
|
on for conversations and prompt and completion pairs |
SFT only: the loss on the answers alone, by TRL’s assistant-only or completion-only loss, or by |
|
TRL’s default |
GRPO only: completions per prompt (TRL’s |
|
TRL’s default |
GRPO only: tokens per completion; less than |
|
TRL’s defaults |
GRPO only: the sampling of the completions; |
|
TRL’s default |
GRPO only: arguments for the chat template, such as |
|
unset: reported only |
SFT only: the largest share of examples whose answer tokens |
|
every row weighs 1 |
SFT only: a weight above 0 for the loss of the rows of each origin kind ( |
Recipe.from_dict(data) and recipe.to_dict() read and write it as data.
Overrides
m2.Overrides(…): the hardware choices, set by hand. A field left as None is auto mode’s. Meep² still measures, and the plan shows its own choice beside yours.
| Field | Auto mode | Meaning |
|---|---|---|
|
FlashAttention 2 where it runs the model, else |
The attention implementation |
|
on |
Liger’s kernels and fused loss. |
|
where the attention keeps the examples apart |
SFT only: examples laid end to end without padding. |
|
where the model’s MLP has a fused kernel |
The fused LoRA kernels. |
|
where it pays |
Each decoder layer compiled with |
|
as the measured memory needs |
How many layers compute their whole forward again in the backward pass. Given alone, the other layers keep every activation. |
|
as the measured memory needs |
How many layers recompute only their cheap operations, keeping the outputs of their products. Given alone, as few layers as the memory needs recompute whole beside them. See Gradient checkpointing. |
|
the largest that fits |
Examples per forward pass. It must divide the examples per update of each process. With a token budget, rows of |
|
off |
GRPO only: the completions generated by vLLM inside the training process (TRL’s colocate mode). Needs TRL 1.16 or later and vLLM; refused with |
Overrides.from_dict(data) and overrides.to_dict() read and write it as data.
Plan
job.plan() returns an m2.Plan. str(plan) (and print(plan)) gives the readable plan: one line for each decision, with its reason. plan.to_dict() gives the same as data: model, task, method, device, free_bytes, decisions, seconds_per_step and estimated_seconds (the timed time per step, and for the planned steps with the evaluations' estimated time; None when not timed), and peak_gpu_bytes, peak_host_bytes and memory_request_bytes (the GPU’s peak reserved memory over the timed steps and the evaluation pass, None on the CPU; the process’s peak resident memory; the memory a job’s limit must cover, 10% over the peak, the GPU’s added on a GPU that shares the system’s memory when the host figure does not count it; all None when not timed). See The memory of the training. plan["checkpointed_layers"] returns one decision by its name.
Each decision has name, label, value, text, reason and by: "auto" for auto mode’s choice, "user" for yours. For yours, auto_value and auto_text say what auto mode would have chosen.
Result
job.train() returns an m2.Result.
| Name | Meaning |
|---|---|
|
The updates the training made. |
|
How long the training took. |
|
The last loss the trainer logged. |
|
The most GPU memory PyTorch allocated during training; |
|
The job’s output folder. |
|
|
|
The latest checkpoint in the output folder; |
|
Each evaluation, |
|
Writes the adapter; with |
|
The fields above as data. |
MeepLanguageModel.from_pretrained
The lower-level calls start here. A training job uses them underneath; use them directly for what a job does not cover, or for code coming from Unsloth (Coming from Unsloth).
Returns (model, tokenizer).
| Argument | Default | Meaning |
|---|---|---|
|
required |
A Hugging Face model id or a local folder. |
|
|
Recorded on the model as |
|
bfloat16, else float16 |
The dtype of the weights, or the compute dtype with |
|
|
QLoRA: NF4 weights with double quantization. Needs the |
|
|
The frozen linear layers of the decoder held as FP8 (e4m3, one scale per tensor) once the adapters are added, their products on the FP8 tensor cores; on a mixture of experts, the expert stacks that carry adapters too, with one scale per expert, dequantized for each product. The routers stay in BF16. Not with |
|
|
Padding-free batches need |
|
|
Liger Kernel’s fused layers and loss. A model type Liger has no patches for is refused unless |
|
|
The chunk size factor of Liger’s fused loss ( |
|
the process’s GPU |
|
|
|
The dtype the 4-bit weights are packed in; |
|
|
Passed to Transformers. |
|
|
|
|
Accepted and not used: they configure only |
Other keyword arguments go to Transformers' AutoModelForCausalLM.from_pretrained. The tokenizer is loaded with legacy=False; when it has no pad token, the end-of-sequence token stands as one.
It prints a banner of what the model trains with; MEEPSQUARED_QUIET=1 in the environment turns it off.
MeepLanguageModel.get_peft_model
Returns a PEFT model. Takes only a model loaded by Meep²'s from_pretrained.
| Argument | Default | Meaning |
|---|---|---|
|
required |
The model from |
|
|
The adapter’s rank, scale and dropout. |
|
the seven projections |
|
|
|
As in PEFT’s |
|
|
Which layers compute their forward again in the backward pass (non-reentrant): |
|
|
The seed of the adapter’s initial weights. |
|
|
Axolotl’s fused LoRA kernels: |
|
|
Each decoder layer compiled as one region with |
Other keyword arguments go to PEFT’s LoraConfig, for example target_parameters; see Mixture of experts.
The model returned has save_pretrained_merged(directory, tokenizer=None, save_method="merged_16bit") and push_to_hub_merged(repo_id, tokenizer=None, save_method="merged_16bit", token=None, private=None, commit_message="Upload model"); see Saving.
MeepLanguageModel.for_inference and for_training
for_inference(model) puts the model in evaluation mode, turns gradient checkpointing off, uncompiles the layers and turns the key-value cache on. for_training(model, use_gradient_checkpointing=None) turns them back as get_peft_model set them, or as given; an "auto" setting keeps the layout the trainer measured, and layers the trainer compiled are compiled again. So generating between job.plan() and job.train() leaves the model as the plan says.
MeepModel
MeepModel.from_pretrained is MeepLanguageModel.from_pretrained. MeepModel.get_peft_model chooses the targets by kind of module when target_modules is not given:
| Argument | Default | Meaning |
|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
fsdp_config
m2.fsdp_config(model, reshard_after_forward=True, cpu_ram_efficient_loading=False) returns SFTConfig’s `fsdp and fsdp_config for FSDP2, sharded by the model’s decoder layer class. See Several GPUs.
train_on_responses_only
meepsquared.chat_templates.train_on_responses_only(trainer, instruction_part, response_part) returns the trainer, its data collator replaced by one that keeps the loss on the tokens after each response_part up to the next instruction_part. See LoRA and QLoRA.
meepsquared.chat_templates.turn_markers(tokenizer) returns (instruction_part, response_part) read from the tokenizer’s chat template, as a training job finds them: the text the template writes before a user turn, and before an answer. It raises ValueError, saying why, for a template they cannot be read from.