# Mooncake Store

The vLLM Project's Speculators training library now supports Kimi K3, a 2.8-trillion-parameter model, and produced a five-layer, five-billion-parameter DSpark draft model that combines parallel block drafting with sequential correction. On math-reasoning workloads, single-stream interactivity rises from about 110 to about 435 tok/s/user, output throughput reaches up to about 3.5x at matched interactivity under concurrent load, and the macro-average acceptance length across nine evaluation domains is 4.11 tokens per verification round.

The vLLM Project's Speculators training library now supports Kimi K3, a 2.8-trillion-parameter model, and produced a five-layer, five-billion-parameter DSpark draft model that combines parallel block drafting with sequential correction. On math-reasoning workloads, single-stream interactivity rises from about 110 to about 435 tok/s/user, output throughput reaches up to about 3.5x at matched interactivity under concurrent load, and the macro-average acceptance length across nine evaluation domains is 4.11 tokens per verification round.

Coding agents resend the same long conversation, so whether the server still holds that history outside GPU memory decides how many tokens each turn must recompute. AMD's open-source ATOM inference engine, which places prefill and decode on separate GPUs, now connects to Mooncake Store to merge both halves of host memory into one KV pool that prefill saves to and restores from. On four AMD Instinct MI350X servers, the shared pool reached about twice the throughput of per-server pools at 96 and 112 concurrent sessions, with KV reuse held at 94–96%.

Coding agents resend the same long conversation, so whether the server still holds that history outside GPU memory decides how many tokens each turn must recompute. AMD's open-source ATOM inference engine, which places prefill and decode on separate GPUs, now connects to Mooncake Store to merge both halves of host memory into one KV pool that prefill saves to and restores from. On four AMD Instinct MI350X servers, the shared pool reached about twice the throughput of per-server pools at 96 and 112 concurrent sessions, with KV reuse held at 94–96%.