Xincheng Wei

The work introduces Neighborhood OPSD (N-OPSD): in on-policy self-distillation, local parameter perturbations of the reference-conditioned teacher yield a set of frozen experts; offline greedy selection builds a complementary pool by rewarding filtered reference-token gains beyond the pool's current best, and online routing uses MaxPeak to pick an anchor token and a quantile rule to pick the level of support among anchor-consistent experts, improving the three-benchmark Average@12 over OPSD by 2.75, 1.67, and 1.94 points on Qwen3-1.7B, 4B, and 8B, with only the distilled student needed at inference.

The work introduces Neighborhood OPSD (N-OPSD): in on-policy self-distillation, local parameter perturbations of the reference-conditioned teacher yield a set of frozen experts; offline greedy selection builds a complementary pool by rewarding filtered reference-token gains beyond the pool's current best, and online routing uses MaxPeak to pick an anchor token and a quantile rule to pick the level of support among anchor-consistent experts, improving the three-benchmark Average@12 over OPSD by 2.75, 1.67, and 1.94 points on Qwen3-1.7B, 4B, and 8B, with only the distilled student needed at inference.