KVCMAS shares multi-agent KV caches via low-rank anchor pools and chained correction, delivering a 2.0 TTFT speedup over non-shared inference at 32K shared tokens and 8 QPS while cutting peak GPU memory to about 1/3.7 of KVComm
The work presents KVCMAS, an online KV cache correction framework for prompt-specialized multi-agent systems that represents cross-agent cache deviations as compact low-rank states and chains corrections along the agent workflow without an additional context-free reference prefill, matching or improving the accuracy of prior KV cache sharing methods across language and vision-language workloads while achieving a 2.0 TTFT speedup over inference without KV cache sharing at 32K shared tokens and 8 QPS and reducing peak GPU memory by up to 3.7 relative to the prior correction method KVComm.