# Self-Distillation

Online Rubric Self-Distillation (ORSD) has the current policy generate natural-language process rubrics from differences among a group of its own trajectories, lets a frozen verifier pick the bundle whose rubric satisfaction best correlates with task success, and uses that bundle as privileged context: the log-likelihood gap for the same tokens with and without the rubric becomes a token-level correction to the trajectory-level GRPO advantage. On ALFWorld and WebShop, Qwen3-1.7B and Qwen3-4B reach the best overall results among the compared methods under the same environment-interaction budget.

Online Rubric Self-Distillation (ORSD) has the current policy generate natural-language process rubrics from differences among a group of its own trajectories, lets a frozen verifier pick the bundle whose rubric satisfaction best correlates with task success, and uses that bundle as privileged context: the log-likelihood gap for the same tokens with and without the rubric becomes a token-level correction to the trajectory-level GRPO advantage. On ALFWorld and WebShop, Qwen3-1.7B and Qwen3-4B reach the best overall results among the compared methods under the same environment-interaction budget.

Two self-distillation methods lose or reverse their reported gains on unseen problems across code, science, tool use and math, and the math improvement comes mainly from transferring thinking-mode behavior rather than from privileged information such as reference solutions.

Two self-distillation methods lose or reverse their reported gains on unseen problems across code, science, tool use and math, and the math improvement comes mainly from transferring thinking-mode behavior rather than from privileged information such as reference solutions.

ClawLoop closes task synthesis, training, and failure diagnosis into one loop so harness-based agents improve without an external teacher model, beating the one-shot ClawGym pipeline on SkillsBench, Claw-Eval-Live, and ClawGym-Bench.

ClawLoop closes task synthesis, training, and failure diagnosis into one loop so harness-based agents improve without an external teacher model, beating the one-shot ClawGym pipeline on SkillsBench, Claw-Eval-Live, and ClawGym-Bench.

UltraWorld presents a self-distillation recipe that first adapts clinical ultrasound videos into a generator conditioned on reference images and anatomical masks, then synthesizes action-video pairs by sampling masks along programmable trajectories through 3D anatomy, distilling the generator into a world model that predicts future observations from local observations and actions alone, and introduces the Acoustic Sampling Map representing probe poses and imaging settings as pixel-wise 3D sampling positions, beam directions, and depths; experiments show improved prediction fidelity and action following, with mean terminal distance and orientation error reduced by 29% and 38% versus visual servoing across nine simulated closed-loop local planning episodes.

UltraWorld presents a self-distillation recipe that first adapts clinical ultrasound videos into a generator conditioned on reference images and anatomical masks, then synthesizes action-video pairs by sampling masks along programmable trajectories through 3D anatomy, distilling the generator into a world model that predicts future observations from local observations and actions alone, and introduces the Acoustic Sampling Map representing probe poses and imaging settings as pixel-wise 3D sampling positions, beam directions, and depths; experiments show improved prediction fidelity and action following, with mean terminal distance and orientation error reduced by 29% and 38% versus visual servoing across nine simulated closed-loop local planning episodes.

The work introduces the notion of Mathematical Primitive and a benchmark that evaluates mathematical reasoning along four dimensions—Discovery, Generation, Digestion, and Execution—and a systematic diagnosis shows that solution accuracy masks distinct capability profiles, that primitives unlock substantial latent execution capacity, and that Discovery is the dominant bottleneck; building on the finding that discovery-limited failures are particularly amenable to repair, it proposes a primitive-privileged self-distillation framework that selectively transfers primitive-guided reasoning into the student model and consistently improves mathematical reasoning over baselines across model scales and challenging benchmarks.

The work introduces the notion of Mathematical Primitive and a benchmark that evaluates mathematical reasoning along four dimensions—Discovery, Generation, Digestion, and Execution—and a systematic diagnosis shows that solution accuracy masks distinct capability profiles, that primitives unlock substantial latent execution capacity, and that Discovery is the dominant bottleneck; building on the finding that discovery-limited failures are particularly amenable to repair, it proposes a primitive-privileged self-distillation framework that selectively transfers primitive-guided reasoning into the student model and consistently improves mathematical reasoning over baselines across model scales and challenging benchmarks.

Taking an entropic perspective on knowledge distillation, the work proves that forward KL inflates the student's entropy above the teacher's, with cross-entropy training as a special case yielding an identity verified quantitatively in pretraining and supervised finetuning; reverse KL deflates entropy until the student-teacher gap grows too large, interpolating between the two changes entropy smoothly early in training but abruptly at convergence; the lower entropy of on-policy distillation comes from token-level reverse KL rather than on-policy sampling, so the divergence acts as an implicit entropy regularizer, most clearly in self-distillation where the best divergence hyperparameters compensate for the entropy deflation caused by conditioning on privileged information.

Taking an entropic perspective on knowledge distillation, the work proves that forward KL inflates the student's entropy above the teacher's, with cross-entropy training as a special case yielding an identity verified quantitatively in pretraining and supervised finetuning; reverse KL deflates entropy until the student-teacher gap grows too large, interpolating between the two changes entropy smoothly early in training but abruptly at convergence; the lower entropy of on-policy distillation comes from token-level reverse KL rather than on-policy sampling, so the divergence acts as an implicit entropy regularizer, most clearly in self-distillation where the best divergence hyperparameters compensate for the entropy deflation caused by conditioning on privileged information.

The work introduces Pivot-SD, an offline self-distillation framework that selects high-impact commitments (pivots) during denoising via an information-gain metric measuring uncertainty reduction over remaining masked positions, trains pivots from successful trajectories with cross-entropy and pivots from failed trajectories with targeted unlikelihood while leaving the rest of the failed trajectory untouched, and with only 200 questions and four rollouts each improves LLaDA-8B-Instruct over full-sequence SFT and budget-matched diffusion RL baselines across math and code benchmarks.

The work introduces Pivot-SD, an offline self-distillation framework that selects high-impact commitments (pivots) during denoising via an information-gain metric measuring uncertainty reduction over remaining masked positions, trains pivots from successful trajectories with cross-entropy and pivots from failed trajectories with targeted unlikelihood while leaving the rest of the failed trajectory untouched, and with only 200 questions and four rollouts each improves LLaDA-8B-Instruct over full-sequence SFT and budget-matched diffusion RL baselines across math and code benchmarks.