Xu Wan

The work defines the length-scaling tax (LST) as the excess response length that RL post-training adds on already-solved queries without a matching accuracy gain, and proposes Length Self-Distillation (LSD), which routes solved prompt groups to on-policy distillation while keeping the original RLVR objective for unsolved groups, using an exponential moving average of the same policy lineage as teacher with no external model; LSD reduces LST from 19.0% to -3.7% on single-turn reasoning and from 31.4% to 13.7% on multi-turn agentic tasks while matching or improving average Pass@1 over RL.

The work defines the length-scaling tax (LST) as the excess response length that RL post-training adds on already-solved queries without a matching accuracy gain, and proposes Length Self-Distillation (LSD), which routes solved prompt groups to on-policy distillation while keeping the original RLVR objective for unsolved groups, using an exponential moving average of the same policy lineage as teacher with no external model; LSD reduces LST from 19.0% to -3.7% on single-turn reasoning and from 31.4% to 13.7% on multi-turn agentic tasks while matching or improving average Pass@1 over RL.