Hao Kang

For coding, math and computer-use agent workloads, Evolving Infra decouples compute allocation between prefill and decode from KV cache management: a node switches phase without a restart, and its KV stays readable from a shared pool. On up to 256 GB300 GPUs it cuts mean end-to-end latency and P95 time to first token by up to 4.1 and 14 relative to NVIDIA Dynamo, and a phase switch takes 6.2 seconds on average against 754 seconds for a Dynamo relaunch.

For coding, math and computer-use agent workloads, Evolving Infra decouples compute allocation between prefill and decode from KV cache management: a node switches phase without a restart, and its KV stays readable from a shared pool. On up to 256 GB300 GPUs it cuts mean end-to-end latency and P95 time to first token by up to 4.1 and 14 relative to NVIDIA Dynamo, and a phase switch takes 6.2 seconds on average against 754 seconds for a Dynamo relaunch.