# Kimi K3

Baseten moved the parent copy of its delta weight sync back onto the GPU, cutting a roughly 770 GB GLM-5.3 checkpoint sync from 36 seconds with a 6-second request pause to 15 seconds with a 0.8-second pause, and raising throughput by as much as 3.2x. The earlier design kept the full parent in host memory, which on an 8×B300 node could not coexist with a 1.1 TB host KV cache, so the cache shrank to 275 GB. The new path reads only changed GPU positions and verifies updates with delta checksums before writing.

Baseten moved the parent copy of its delta weight sync back onto the GPU, cutting a roughly 770 GB GLM-5.3 checkpoint sync from 36 seconds with a 6-second request pause to 15 seconds with a 0.8-second pause, and raising throughput by as much as 3.2x. The earlier design kept the full parent in host memory, which on an 8×B300 node could not coexist with a 1.1 TB host KV cache, so the cache shrank to 275 GB. The new path reads only changed GPU positions and verifies updates with delta checksums before writing.

Open-weight Kimi K3 is available on Fireworks for inference and training from day one, which the release presents as frontier-level capability at a fraction of closed-model cost. On Arena's Frontend Code Arena it scores 1,679 points, ahead of Fable 5, making it the first open model to sit ahead of every closed one; it ranks third worldwide on the Artificial Analysis Intelligence Index at 57. Fireworks also offers US-only inference endpoints with zero data retention by default and a private preview of serverless training on a shared GPU pool.

Open-weight Kimi K3 is available on Fireworks for inference and training from day one, which the release presents as frontier-level capability at a fraction of closed-model cost. On Arena's Frontend Code Arena it scores 1,679 points, ahead of Fable 5, making it the first open model to sit ahead of every closed one; it ranks third worldwide on the Artificial Analysis Intelligence Index at 57. Fireworks also offers US-only inference endpoints with zero data retention by default and a private preview of serverless training on a shared GPU pool.

Kimi K3's decode path in vLLM is now a unified CuTe DSL implementation built with agent assistance, fusing attention, MoE and cross-GPU communication. On 16 NVIDIA GB300 GPUs with DSpark at synthetic acceptance length 6, decode reaches 842 tokens/s, 1.73x stock vLLM under the same speculative setup, and without speculation TP16 keeps 1.62-1.71x at batches 16-64. A proposed fast-track process pairs such work with isolated changes, bounded hardware scope and both kernel and model-level validation.

Kimi K3's decode path in vLLM is now a unified CuTe DSL implementation built with agent assistance, fusing attention, MoE and cross-GPU communication. On 16 NVIDIA GB300 GPUs with DSpark at synthetic acceptance length 6, decode reaches 842 tokens/s, 1.73x stock vLLM under the same speculative setup, and without speculation TP16 keeps 1.62-1.71x at batches 16-64. A proposed fast-track process pairs such work with isolated changes, bounded hardware scope and both kernel and model-level validation.

On the public AgentX agent workload, an eight-GPU AMD Instinct MI355X system serving Kimi K3 with vLLM reaches about 7,567 total tokens/s/GPU at 50 tokens/s/user P90 interactivity, 24.4% above NVIDIA HGX B300's 6,083; under SemiAnalysis's modeled $1.50 versus $2.26 per GPU-hour, that is 18.16 million versus 9.69 million tokens per dollar of TCO, 87.4% more. The result comes from sustained AMD and vLLM community engineering across attention, recurrent state, MoE and caching rather than a single change.

On the public AgentX agent workload, an eight-GPU AMD Instinct MI355X system serving Kimi K3 with vLLM reaches about 7,567 total tokens/s/GPU at 50 tokens/s/user P90 interactivity, 24.4% above NVIDIA HGX B300's 6,083; under SemiAnalysis's modeled $1.50 versus $2.26 per GPU-hour, that is 18.16 million versus 9.69 million tokens per dollar of TCO, 87.4% more. The result comes from sustained AMD and vLLM community engineering across attention, recurrent state, MoE and caching rather than a single change.