# GLM-5.3

Baseten moved the parent copy of its delta weight sync back onto the GPU, cutting a roughly 770 GB GLM-5.3 checkpoint sync from 36 seconds with a 6-second request pause to 15 seconds with a 0.8-second pause, and raising throughput by as much as 3.2x. The earlier design kept the full parent in host memory, which on an 8×B300 node could not coexist with a 1.1 TB host KV cache, so the cache shrank to 275 GB. The new path reads only changed GPU positions and verifies updates with delta checksums before writing.

Baseten moved the parent copy of its delta weight sync back onto the GPU, cutting a roughly 770 GB GLM-5.3 checkpoint sync from 36 seconds with a 6-second request pause to 15 seconds with a 0.8-second pause, and raising throughput by as much as 3.2x. The earlier design kept the full parent in host memory, which on an 8×B300 node could not coexist with a 1.1 TB host KV cache, so the cache shrank to 275 GB. The new path reads only changed GPU positions and verifies updates with delta checksums before writing.

Serving GLM-5.3's long contexts on 16 H200s runs into two limits: KV cache memory and prefill speed. On the prefill side, each prompt is split across eight GPUs by token (PCP) and the KV cache is sharded by token (DCP), so a 1M-token request needs only 6.7 GiB per GPU; on the decode side, Hybrid HiSparse keeps KV in host memory. On the AgentX benchmark, PCP8 × DCP8 prefill with HiSparse decode gives 1.7–3.2× the per-user speed of a DP8 deployment at the same throughput, and it is the only tested configuration that serves 1M-token contexts.

Serving GLM-5.3's long contexts on 16 H200s runs into two limits: KV cache memory and prefill speed. On the prefill side, each prompt is split across eight GPUs by token (PCP) and the KV cache is sharded by token (DCP), so a 1M-token request needs only 6.7 GiB per GPU; on the decode side, Hybrid HiSparse keeps KV in host memory. On the AgentX benchmark, PCP8 × DCP8 prefill with HiSparse decode gives 1.7–3.2× the per-user speed of a DP8 deployment at the same throughput, and it is the only tested configuration that serves 1M-token contexts.