OnlineQAT uses on-policy distillation on student-generated prefixes to lift Qwen3-1.7B to a 57.28 six-task average at W3A16, 2.90 points above ReasoningQAT
OnlineQAT is a two-stage ultra-low-bit recovery framework that first obtains a usable W2/W3 initialization through block-wise quantization-aware training, then performs on-policy distillation on responses sampled from the quantized student, with a frozen full-precision teacher supplying a sampled reverse-KL signal at each visited prefix; on Qwen3-1.7B it reaches six-task averages of 57.28 at W3A16 and 32.52 at W2A16, improving over ReasoningQAT by 2.90 and 0.44 points, suggesting that student-visited states offer a recovery signal beyond fixed-completion training, particularly at three bits.