When a long-context request dies mid-transfer, its KV cache can land in someone else's output: SGLang now holds slots until writes drain

In disaggregated prefill/decode serving, a request that is aborted can still have KV cache writes in flight, and those writes can land in another request's slots so the later request decodes on someone else's context. Earlier implementations returned the decode-side destination slots to the allocator at termination without waiting for a launched write to finish; the corruption raised no exception and logged no error, was rare in typical workloads, and became far more likely with long contexts. Cloudflare reports that @cf/zai-org/glm-5.3 on Workers AI was affected between August 27 and September 9, 2026, and that its investigation found no other Workers AI model affected; its configured context length was 1,048,576 tokens while its prefill KV cache pool could hold 681,280 tokens.

Decode no longer frees a request's slots at abort time; it waits until each notified prefill rank confirms its launched writes have drained, or until a bounded timeout expires. The mechanism is deferred KV cache release: on abort, decode sends ABORT to every prefill rank, each rank replies ABORT_ACK once its dequeue count reaches zero, and decode releases only after all acknowledgments arrive, with the timeout as fallback. On a 2×H200 1P1D pair with Mooncake over RDMA, a one-sided abort freed the donor's 430 slots and the victim preallocated [1..105] inside them while the donor's launched write landed mid-decode; with the hold enabled, the victim's output was identical to the baseline byte for byte.

The admission path was tightened as well: a rejected request now returns an error before entering the disaggregated handoff instead of completing a bootstrap handshake and a one-token forward pass. Previously a rejected request still reported itself as live, completed a bootstrap handshake, reserved a metadata buffer and ran a forward pass before being retired; if both workers rejected a request, decode timed out after 300 seconds. #38935, merged on September 16, 2026, retires a request when it enters PrefillBootstrapQueue.add or DecodePreallocQueue.add; when both workers reject a request, decode now returns 400 at once.

The protection targets deployments with disaggregated prefill/decode enabled and a KV cache transfer backend (Mooncake, NIXL, MORI or a custom CommonKVManager backend); a single-instance deployment without that cross-engine transfer path is not affected by this race. Default configurations were affected from the v0.4.x line until v0.5.21, or v0.5.22 for MORI. Operators are advised to upgrade to v0.5.21 or later and to run a router that cancels both halves of a pair.

Timeout-based release is safe only if outstanding writes have stopped by then; expiry alone does not confirm they have drained. When prefill and decode use different acknowledgement formats, the drain handshake can fail and releases fall back to the configured timeout. After a failed Mooncake send, every failure costs the full hold until Mooncake can report whether a failed batch has drained.