# Low-rank KV cache compression

Reasoning vision-language models must hold both multimodal context and a growing reasoning trace in their KV cache, and existing low-rank compression calibrates on static inputs, so its divergence from the dense model grows with generation length. TraceKV instead calibrates on reasoning traces the dense model generates itself, weights each KV state by its sensitivity to policy divergence before the SVD, and allocates ranks globally. Across four models including Qwen2.5-VL-7B, a 70% rank ratio with 4-bit quantization gives 73.41% overall accuracy versus 73.58% for the dense model, with the KV cache reduced 5.64 times.

Reasoning vision-language models must hold both multimodal context and a growing reasoning trace in their KV cache, and existing low-rank compression calibrates on static inputs, so its divergence from the dense model grows with generation length. TraceKV instead calibrates on reasoning traces the dense model generates itself, weights each KV state by its sensitivity to policy divergence before the SVD, and allocates ranks globally. Across four models including Qwen2.5-VL-7B, a 70% rank ratio with 4-bit quantization gives 73.41% overall accuracy versus 73.58% for the dense model, with the KV cache reduced 5.64 times.

The work proposes iS-KV, an online low-rank KV-cache compression method for long-horizon reasoning that keeps a recent window exact, incrementally folds older states into bounded-rank representations, and synchronizes historical coordinates as the low-rank basis evolves to maintain representation consistency; on DeepSeek-R1-Distill-Llama-8B it reaches 82.6% accuracy at 4.06-fold persistent-KV compression (original model 83.6%), and on Qwen3-8B it reaches 89.2% accuracy at 5.64-fold compression, consistently outperforming token-eviction baselines under matched memory budgets.

The work proposes iS-KV, an online low-rank KV-cache compression method for long-horizon reasoning that keeps a recent window exact, incrementally folds older states into bounded-rank representations, and synchronizes historical coordinates as the low-rank basis evolves to maintain representation consistency; on DeepSeek-R1-Distill-Llama-8B it reaches 82.6% accuracy at 4.06-fold persistent-KV compression (original model 83.6%), and on Qwen3-8B it reaches 89.2% accuracy at 5.64-fold compression, consistently outperforming token-eviction baselines under matched memory budgets.