# On-Policy Calibration

Reasoning vision-language models must hold both multimodal context and a growing reasoning trace in their KV cache, and existing low-rank compression calibrates on static inputs, so its divergence from the dense model grows with generation length. TraceKV instead calibrates on reasoning traces the dense model generates itself, weights each KV state by its sensitivity to policy divergence before the SVD, and allocates ranks globally. Across four models including Qwen2.5-VL-7B, a 70% rank ratio with 4-bit quantization gives 73.41% overall accuracy versus 73.58% for the dense model, with the KV cache reduced 5.64 times.

Reasoning vision-language models must hold both multimodal context and a growing reasoning trace in their KV cache, and existing low-rank compression calibrates on static inputs, so its divergence from the dense model grows with generation length. TraceKV instead calibrates on reasoning traces the dense model generates itself, weights each KV state by its sensitivity to policy divergence before the SVD, and allocates ranks globally. Across four models including Qwen2.5-VL-7B, a 70% rank ratio with 4-bit quantization gives 73.41% overall accuracy versus 73.58% for the dense model, with the KV cache reduced 5.64 times.

Deciding which change is worth trying when a language model improves a language model has depended on costly training experiments. LM2LM uses a frozen LLM as a research world model that predicts a proposal's gain from real experimental records in context, turning those predictions into reinforcement-learning rewards, and periodically executes the current policy's top-rated proposals to write measured outcomes back into the records. Across eight autoregressive and diffusion language-model development environments, Qwen3.8-27B trained this way outperforms GPT-6-sol on held-out tasks, even when GPT-6-sol is given 300 real experimental records in context, at about 33% lower data-generation cost.

Deciding which change is worth trying when a language model improves a language model has depended on costly training experiments. LM2LM uses a frozen LLM as a research world model that predicts a proposal's gain from real experimental records in context, turning those predictions into reinforcement-learning rewards, and periodically executes the current policy's top-rated proposals to write measured outcomes back into the records. Across eight autoregressive and diffusion language-model development environments, Qwen3.8-27B trained this way outperforms GPT-6-sol on held-out tasks, even when GPT-6-sol is given 300 real experimental records in context, at about 33% lower data-generation cost.