# Prefill/Decode Disaggregation

DeepSeek open-sourced DeepInfer, the inference engine behind its own service, along with its kernels; SGLang then brought the Mega kernel series, DSpark speculative decoding, SWA bounded replay and both Engram placements into its mainline and compared against DeepInfer on the same hardware using DeepInfer's own benchmarks. On 4x GB300 and 4x B300, SGLang reaches up to 1.04x DeepInfer's prefill throughput, within 1% at the remaining points; on 64x GB300 its decode step is 1.1% to 7.4% shorter from 16 to 256 requests per GPU and on par at 8. SGLang also ships the KV cache system, prefill/decode disaggregation and cross-replica load balancing that DeepInfer's release notes list as not included.

DeepSeek open-sourced DeepInfer, the inference engine behind its own service, along with its kernels; SGLang then brought the Mega kernel series, DSpark speculative decoding, SWA bounded replay and both Engram placements into its mainline and compared against DeepInfer on the same hardware using DeepInfer's own benchmarks. On 4x GB300 and 4x B300, SGLang reaches up to 1.04x DeepInfer's prefill throughput, within 1% at the remaining points; on 64x GB300 its decode step is 1.1% to 7.4% shorter from 16 to 256 requests per GPU and on par at 8. SGLang also ships the KV cache system, prefill/decode disaggregation and cross-replica load balancing that DeepInfer's release notes list as not included.

Serving GLM-5.3's long contexts on 16 H200s runs into two limits: KV cache memory and prefill speed. On the prefill side, each prompt is split across eight GPUs by token (PCP) and the KV cache is sharded by token (DCP), so a 1M-token request needs only 6.7 GiB per GPU; on the decode side, Hybrid HiSparse keeps KV in host memory. On the AgentX benchmark, PCP8 × DCP8 prefill with HiSparse decode gives 1.7–3.2× the per-user speed of a DP8 deployment at the same throughput, and it is the only tested configuration that serves 1M-token contexts.

Serving GLM-5.3's long contexts on 16 H200s runs into two limits: KV cache memory and prefill speed. On the prefill side, each prompt is split across eight GPUs by token (PCP) and the KV cache is sharded by token (DCP), so a 1M-token request needs only 6.7 GiB per GPU; on the decode side, Hybrid HiSparse keeps KV in host memory. On the AgentX benchmark, PCP8 × DCP8 prefill with HiSparse decode gives 1.7–3.2× the per-user speed of a DP8 deployment at the same throughput, and it is the only tested configuration that serves 1M-token contexts.