Async RL squeezes context into fewer decode steps; INT4 KV cache speeds rollouts but loses accuracy, bounded INT8 protection recovers it
Sustained concurrency in asynchronous RLVR rollout concentrates context processing into fewer decode steps, making the KV cache rather than weights the dominant cost. KV4-RL stores the full context in calibration-free INT4 and protects only attention sinks and recent context in bounded INT8; across full-parameter GRPO training on Qwen3-4B-Base and Qwen3-8B-Base, its six-benchmark mean accuracy stays within 0.5 percentage points of BF16 rollout while substantially accelerating rollout and full iterations.