# Compressed KV Caching

Measuring attention-head behavior across three models (1.5B–8B) and three regimes (needle retrieval, long chain-of-thought, multi-turn recall), the authors find most heads change their reading behavior at least once during generation and introduce WakeKV, a reactive residency policy that moves cooling heads to a recoverable CPU reservoir instead of freezing or permanently evicting their state; at matched memory or budget WakeKV consistently improves miss rate over frozen classification and destructive eviction, and a FlexiCache/vLLM implementation on Mistral-7B improves throughput while retaining LongBench quality.

Measuring attention-head behavior across three models (1.5B–8B) and three regimes (needle retrieval, long chain-of-thought, multi-turn recall), the authors find most heads change their reading behavior at least once during generation and introduce WakeKV, a reactive residency policy that moves cooling heads to a recoverable CPU reservoir instead of freezing or permanently evicting their state; at matched memory or budget WakeKV consistently improves miss rate over frozen classification and destructive eviction, and a FlexiCache/vLLM implementation on Mistral-7B improves throughput while retaining LongBench quality.

The authors present SoloQ, which uses a K-RPBH rotation to map weights and activations into a predictable near-Gaussian basis plus a scale-only bias correction for calibration-free 4-bit quantization, and quantizes the KV cache at block commit for block-diffusion models; across LLaDA-Base, LLaDA-1.5, Dream-7B, Fast-dLLM v2 and Nemotron-Labs-Diffusion, SoloQ-C and SoloQ-N retain accuracy under W4A4 and W4A4KV4 and exceed calibration-based baselines such as DLLMQuant++, STaR-Quant and FAIR-Calib on average, with the NVFP4 path also cutting peak memory and speeding up end-to-end inference.

The authors present SoloQ, which uses a K-RPBH rotation to map weights and activations into a predictable near-Gaussian basis plus a scale-only bias correction for calibration-free 4-bit quantization, and quantizes the KV cache at block commit for block-diffusion models; across LLaDA-Base, LLaDA-1.5, Dream-7B, Fast-dLLM v2 and Nemotron-Labs-Diffusion, SoloQ-C and SoloQ-N retain accuracy under W4A4 and W4A4KV4 and exceed calibration-based baselines such as DLLMQuant++, STaR-Quant and FAIR-Calib on average, with the NVFP4 path also cutting peak memory and speeding up end-to-end inference.