# NIXL EP

Large mixture-of-experts (MoE) inference deployments previously had two scaling options: add a full model replica, or restart into a wider expert-parallel topology. SGLang's elastic expert-parallel scale-up adds GPU ranks to a running deployment, reserving metadata and communication buffers for max_ep_size at launch, admitting joiners through Mooncake PG, and carrying expert traffic over NIXL before committing the new topology. On 8×H100 with DeepSeek-V3-Lite FP8, scale-up reached the first completed request 5.3× to 8.2× faster than restart.

Large mixture-of-experts (MoE) inference deployments previously had two scaling options: add a full model replica, or restart into a wider expert-parallel topology. SGLang's elastic expert-parallel scale-up adds GPU ranks to a running deployment, reserving metadata and communication buffers for max_ep_size at launch, admitting joiners through Mooncake PG, and carrying expert traffic over NIXL before committing the new topology. On 8×H100 with DeepSeek-V3-Lite FP8, scale-up reached the first completed request 5.3× to 8.2× faster than restart.

In vLLM's Elastic Expert Parallelism, resizing a deployment used to stop serving: scaling DeepSeek V4 Pro from 16 to 20 GB200 GPUs paused for 99.62 seconds. The new design splits reconfiguration into background preparation and a short commit, while NIXL EP adds or removes communication ranks in place and keeps captured CUDA graphs valid, cutting the same benchmark's pause to 0.82 seconds on scale-up and 1.96 seconds on scale-down.

In vLLM's Elastic Expert Parallelism, resizing a deployment used to stop serving: scaling DeepSeek V4 Pro from 16 to 20 GB200 GPUs paused for 99.62 seconds. The new design splits reconfiguration into background preparation and a short commit, while NIXL EP adds or removes communication ranks in place and keeps captured CUDA graphs valid, cutting the same benchmark's pause to 0.82 seconds on scale-up and 1.96 seconds on scale-down.