# On-Policy Self-Distillation

In video reasoning the correct answer often hinges on a few visually decisive positions, yet post-training methods such as GRPO assign one coarse score to the whole reasoning trajectory and need multiple rollouts or external judges. VIOS instead has the model audit its own visual reliance: it compares next-token distributions with and without the video, takes the per-token reliance gap between the student and an answer-privileged teacher, and concentrates supervision where the student under-uses visual evidence. Across 12 video-understanding benchmarks, Qwen2.5-VL-7B-SFT rises from 46.35 to 49.95 on average, InternVL2.5-4B and InternVL2.5-8B gain 4.63 and 3.93 points, and training needs a single on-policy rollout with no external reward or judge.

In video reasoning the correct answer often hinges on a few visually decisive positions, yet post-training methods such as GRPO assign one coarse score to the whole reasoning trajectory and need multiple rollouts or external judges. VIOS instead has the model audit its own visual reliance: it compares next-token distributions with and without the video, takes the per-token reliance gap between the student and an answer-privileged teacher, and concentrates supervision where the student under-uses visual evidence. Across 12 video-understanding benchmarks, Qwen2.5-VL-7B-SFT rises from 46.35 to 49.95 on average, InternVL2.5-4B and InternVL2.5-8B gain 4.63 and 3.93 points, and training needs a single on-policy rollout with no external reward or judge.

On-policy self-distillation lets one model act as a problem-only student and as a teacher given a reference solution, but the reference pushes the teacher toward short, answer-conditioned derivations that omit exploration and self-correction. TeachBack injects a trailing slice of the reference reasoning as optional working notes and optimizes preferences in both directions: teacher trajectories should win under the problem-only prompt, while student trajectories reshape future privileged teacher rollouts. Across Qwen3-1.7B, 4B and 8B, math averages rise by 4.3, 2.2 and 2.5 points over base, and Knight-and-Knaves averages by 6.1, 3.9 and 7.5 points.

On-policy self-distillation lets one model act as a problem-only student and as a teacher given a reference solution, but the reference pushes the teacher toward short, answer-conditioned derivations that omit exploration and self-correction. TeachBack injects a trailing slice of the reference reasoning as optional working notes and optimizes preferences in both directions: teacher trajectories should win under the problem-only prompt, while student trajectories reshape future privileged teacher rollouts. Across Qwen3-1.7B, 4B and 8B, math averages rise by 4.3, 2.2 and 2.5 points over base, and Knight-and-Knaves averages by 6.1, 3.9 and 7.5 points.

Image factuality assessment no longer stops at whether a picture is AI-generated: the EviScene multimodal agent refines its own questions during investigation and reaches 79.05% balanced accuracy and 52.26% evidence sufficiency on 1,527 test images.

Image factuality assessment no longer stops at whether a picture is AI-generated: the EviScene multimodal agent refines its own questions during investigation and reaches 79.05% balanced accuracy and 52.26% evidence sufficiency on 1,527 test images.

Two self-distillation methods lose or reverse their reported gains on unseen problems across code, science, tool use and math, and the math improvement comes mainly from transferring thinking-mode behavior rather than from privileged information such as reference solutions.

Two self-distillation methods lose or reverse their reported gains on unseen problems across code, science, tool use and math, and the math improvement comes mainly from transferring thinking-mode behavior rather than from privileged information such as reference solutions.

LoOPSD builds a stronger self-teacher by repeating a searched block of a model's own layers without changing any weights, then distills that teacher's predictions into a standard-path student, lifting Qwen3-1.7B on GSM8K from 42.30% to 79.61% under a 400-token thinking budget.

LoOPSD builds a stronger self-teacher by repeating a searched block of a model's own layers without changing any weights, then distills that teacher's predictions into a standard-path student, lifting Qwen3-1.7B on GSM8K from 42.30% to 79.61% under a 400-token thinking budget.

Vision-language models struggle with spatial reasoning because RGB inputs do not directly supply geometric evidence, and existing fixes either add 3D at inference, paying architecture and latency costs, or train on outcome rewards that supervise only the final answer. GPD renders depth, semantic segmentation and bird's-eye-view layout as compact text, routes the question-relevant subset to the teacher alongside the reference answer, and applies a privileged KL only to incorrect trajectories within GRPO, leaving the deployed model RGB-only. On the 4B backbone it reaches 57.1 on VSI-Bench and 37.6 average across MindCube, SPAR-Bench, MMSI-Bench and ViewSpatial, above GRPO and answer-privileged OPSD.

Vision-language models struggle with spatial reasoning because RGB inputs do not directly supply geometric evidence, and existing fixes either add 3D at inference, paying architecture and latency costs, or train on outcome rewards that supervise only the final answer. GPD renders depth, semantic segmentation and bird's-eye-view layout as compact text, routes the question-relevant subset to the teacher alongside the reference answer, and applies a privileged KL only to incorrect trajectories within GRPO, leaving the deployed model RGB-only. On the 4B backbone it reaches 57.1 on VSI-Bench and 37.6 average across MindCube, SPAR-Bench, MMSI-Bench and ViewSpatial, above GRPO and answer-privileged OPSD.

Multimodal LLMs often report no text in images whose coarse structure carries a string a human recovers by squinting, and a training method that reads the original image alone lifts transcription from 26.50 to 88.61 on production moderation images.

Multimodal LLMs often report no text in images whose coarse structure carries a string a human recovers by squinting, and a training method that reads the original image alone lifts transcription from 26.50 to 88.61 on production moderation images.

The authors introduce SpatialOPSD, an on-policy self-distillation framework that treats verified execution traces from a spatial coding agent as privileged information, letting a frozen teacher initialized from the same checkpoint see the traces while the student sees only raw images, with Repetition-Aware Distillation (repetition masking plus unlikelihood regularization) suppressing verbatim copying; on MindCube-Tiny and ViewSpatial, the 9B student raises average accuracy from 45.9 to 51.9, above SFT at 47.6 and GRPO at 48.2, and retains about 91% of its tool-using teacher SpatialClaw at 57.3 in a single tool-free forward pass.

The authors introduce SpatialOPSD, an on-policy self-distillation framework that treats verified execution traces from a spatial coding agent as privileged information, letting a frozen teacher initialized from the same checkpoint see the traces while the student sees only raw images, with Repetition-Aware Distillation (repetition masking plus unlikelihood regularization) suppressing verbatim copying; on MindCube-Tiny and ViewSpatial, the 9B student raises average accuracy from 45.9 to 51.9, above SFT at 47.6 and GRPO at 48.2, and retains about 91% of its tool-using teacher SpatialClaw at 57.3 in a single tool-free forward pass.