Multi-turn agent distillation stops feeding student reasoning to the teacher: a 4B student beats its 32B teacher on ALFWorld Seen at 43.39%

Teacher supervision in multi-turn agent distillation no longer inherits the student's earlier reasoning and is grounded only in the interaction state, the task instruction plus the record of executed actions and returned observations. Earlier on-policy distillation handed the teacher the student's full history, which includes the student's own reasoning traces. On ALFWorld, replaying the executed action prefix to restore the environment and running paired continuations, the teacher continuation success without inherited reasoning is higher in every anchor range; for Qwen3-4B the paired gain rises from 0.8 percentage points at early anchors to 20.3 points at late anchors.

On the student side, sampling moves to complete reasoning-plus-action responses at the same interaction state, instead of fixing one reasoning trace and resampling only the action. With a fixed reasoning trace, nearly a single action per state entered supervision. Across 64 Qwen3-4B trajectories on ALFWorld, with matched 16-response sets built at each state, 88.5% of fixed-reasoning probes yield just one parsed action, compared with 44.3% of complete-response probes.

State-OPD grounds teacher targets in the interaction state and concentrates the sampling budget on states with high response entropy, while the student still generates from its full history. The reference configuration spread four rollouts per task evenly across turns. Across ALFWorld, ScienceWorld and WebShop and two student sizes, the Qwen3-4B macro-average success rises from 26.07% with the strongest baseline, Vanilla OPD, to 31.74%, and the Qwen3-1.7B student leads its group at 20.76% under the same configuration.

The report covers three text-based interactive environments, ALFWorld, ScienceWorld and WebShop, with Qwen3-1.7B and Qwen3-4B students distilled from a frozen Qwen3-32B teacher, sharing prompts and history-retention settings between training and evaluation. It suits researchers and engineers working on multi-turn agent post-training who care about which part of the context carries teacher supervision. The source states no next-step experiment.

Each configuration has one training run with seed 42, so the reported success rates do not estimate variation across training seeds. The task-paired intervals quantify evaluation uncertainty conditional on fixed checkpoints and are pointwise 95% intervals without a multiple-comparison correction. Ablations and behavioral analyses concentrate on ALFWorld, while the other two environments appear only in the main table.