Reference
Trainer settings
How a training job sets up TRL’s trainers, what TRL’s SFTTrainer and GRPOTrainer take from a Meep² model, and when your config wins.
A training job builds TRL’s SFTTrainer or GRPOTrainer from its plan, and job.trainer() returns it. With the lower-level calls, importing Meep² makes those trainers read their settings from a model Meep² loaded; a setting you give in SFTConfig or GRPOConfig is kept, except where the table says it is refused. A model Meep² did not load trains as TRL alone would train it.
What a training job sets
A training job writes these settings into TRL’s config, from its plan. Each one is a line of the plan, with its reason; Training jobs gives the defaults and their sources.
| TRL setting | From the plan |
|---|---|
|
The micro-batch: the largest share of the effective batch that fits, measured; or |
|
The effective batch divided by the micro-batch and the number of processes. |
|
By the model’s size, or |
|
Constant after a warm-up, or cosine decay to a tenth with |
|
|
|
The job’s |
|
SFT: |
|
GRPO: |
|
SFT: as the model allows, or |
|
SFT: the loss on the answers, as |
|
|
|
GRPO: the recipe’s fields of the same names; TRL’s defaults otherwise. |
|
GRPO with |
|
The job’s |
|
The dtype the model computes in. |
|
|
|
The job’s |
The layers recomputed whole or selectively are set on the model, as the plan measured them or as Overrides(checkpointed_layers=…, selective_layers=…) gives. Other settings are TRL’s defaults. To change one the job does not have, build the trainer with the lower-level calls (see LoRA and QLoRA).
Settings taken from the model
With the lower-level calls, the trainer takes these from a model Meep² loaded. A training job goes through the same path.
| Setting | Trainer | Value, unless your config gives it |
|---|---|---|
|
SFT |
On when |
|
SFT, GRPO |
As |
|
SFT, GRPO |
|
|
SFT, GRPO |
As |
|
SFT, GRPO |
|
|
SFT, GRPO |
|
Settings refused
| Setting | Why |
|---|---|
|
Accelerate turns the BF16 weights to FP32 before sharding, which doubles their memory and the traffic between the GPUs. |
A bitsandbytes optimizer ( |
bitsandbytes cannot update FSDP2’s sharded parameters. Use |
Other adjustments
-
No
nn.DataParallel. One process with a model on one GPU uses that GPU only, whatever number of GPUs it sees. -
max_lengthwith padding-free batches on TRL 1.x. TRL 1.x refusespadding_freewithmax_lengthand no packing. Whether Meep² turns padding-free on or your config givespadding_free=True, Meep² builds the trainer withoutmax_lengthand cuts each example to it as it reaches the data collator, keeping its start, as TRL 0.29 does. -
Fused LoRA after Liger. With the fused LoRA kernels on, they are applied again after the trainer is built, in place of the SwiGLU forward Liger set.
-
A model spread over GPUs. After the trainer is built, each part runs on its own GPU again, and the batch’s token count and the loss move to the GPUs they are needed on.
-
No config at all.
SFTTrainer(model=model, …)withoutargsgets a config named after the model, with the settings above.