# Bi-directional KV transfer

Multi-turn agentic conversations resend their history every turn, and KV cache previously flowed only one way, from prefill to decode, so the decode node's generated cache could not be reused. vLLM adds bi-directional KV transfer, letting the decode node retain and return computed KV blocks while the prefill node decides by a configurable threshold whether to reuse or recompute. On a 1P1D deployment of Amazon EC2 P5en instances with EFA, time to first token fell by up to 1.6x for Qwen3-32B and up to 3.3x for GLM-5-FP8 at the longest prompts, with gains growing by prompt length and depending on workload and configuration.

Multi-turn agentic conversations resend their history every turn, and KV cache previously flowed only one way, from prefill to decode, so the decode node's generated cache could not be reused. vLLM adds bi-directional KV transfer, letting the decode node retain and return computed KV blocks while the prefill node decides by a configurable threshold whether to reuse or recompute. On a 1P1D deployment of Amazon EC2 P5en instances with EFA, time to first token fell by up to 1.6x for Qwen3-32B and up to 3.3x for GLM-5-FP8 at the longest prompts, with gains growing by prompt length and depending on workload and configuration.