# Selective Supervision

ROSS introduces a selective-supervision relearning procedure that takes historical self-generated rollouts saved from domain-specific RL, multi-teacher on-policy distillation (MOPD), or agentic RL, uses an outcome verifier to keep successful trajectories and an LLM reviewer to mark which model-generated spans are worth imitating, then applies loss only to those selected tokens while keeping the full trajectory as context, improving upstream checkpoints through an offline SFT stage without new policy rollouts and raising Qwen3.6-35B-A3B's six-benchmark MOPD average from 58.40% to 62.20% and SWE-bench Verified from 64.20% to 68.40%.

ROSS introduces a selective-supervision relearning procedure that takes historical self-generated rollouts saved from domain-specific RL, multi-teacher on-policy distillation (MOPD), or agentic RL, uses an outcome verifier to keep successful trajectories and an LLM reviewer to mark which model-generated spans are worth imitating, then applies loss only to those selected tokens while keeping the full trajectory as context, improving upstream checkpoints through an offline SFT stage without new policy rollouts and raising Qwen3.6-35B-A3B's six-benchmark MOPD average from 58.40% to 62.20% and SWE-bench Verified from 64.20% to 68.40%.

Using a fixed-context Pass@K analysis, this work shows that degradation after visual-token pruning is not only lost information but also unreliable use of surviving evidence (a representation-utilization gap), and introduces SCOPD and SCOPD+ sparse-context on-policy self-distillation, raising the normalized average over 13 benchmarks at 10% visual-token retention from 86.37% (Vanilla) to 90.49% and 92.43% without architectural changes, extra inference-time computation, or ground-truth answers.

Using a fixed-context Pass@K analysis, this work shows that degradation after visual-token pruning is not only lost information but also unreliable use of surviving evidence (a representation-utilization gap), and introduces SCOPD and SCOPD+ sparse-context on-policy self-distillation, raising the normalized average over 13 benchmarks at 10% visual-token retention from 86.37% (Vanilla) to 90.49% and 92.43% without architectural changes, extra inference-time computation, or ground-truth answers.