Agent Substrate enters the CNCF Sandbox: idle agent sessions are checkpointed out, with the project claiming 10x container density
Synopsis
NVIDIA has donated Agent Substrate, an open-source runtime for running large numbers of sandboxed AI agents on Kubernetes, to the CNCF Sandbox. Agents spend most of their time waiting on a prompt, a tool result or a model response; keeping one Pod per session holds memory for idle sessions, while creating Pods on demand pays a multi-second cold start at every step and loses state. Agent Substrate multiplexes many sessions over a small pool of warm worker Pods, checkpointing idle sessions including memory and filesystem and restoring them onto whichever worker is free, and the project reports 10x higher density than standard containers.
Interpretation
Agent workloads no longer need a dedicated warm Pod per session or a rebuilt sandbox at every step; many sessions can share a small pool of ready worker Pods. Platform teams previously had two choices: keep one Pod warm per agent and pay for compute that is idle almost all the time, or create Pods on demand and pay a cold-start penalty on every step. Creating a Pod on demand goes through the API server, admission, etcd, the scheduler, the kubelet, image pulls and network setup, which takes seconds, far too slow for a sandbox that must appear, run a 10 ms bash command and get out of the way hundreds of times per second.
When a session goes idle it is snapshotted out of its sandbox, its memory and filesystem state are saved, and its worker is freed for other sessions. Waking a session no longer requires a new trip through the API server, scheduler or kubelet; it is restored onto an existing worker instead. Worker Pods remain ordinary Kubernetes Pods, so cluster scheduling, quotas and placement policies still apply, and the sandbox can be gVisor or a microVM.
Inference and execution split the work: Agent Substrate manages the high-frequency lifecycle of agent environments while NVIDIA Dynamo manages the high-performance lifecycle of inference, and the two exchange inference-state signals. Traditional inference routing treats every model call as an independent request, whereas a request inside an agent is one step in a longer loop that often reuses the same system prompt, tool definitions and conversation history. Dynamo can return projected time to first token, projected completion time, request progress, cache readiness and GPU-pool saturation, and Agent Substrate uses that feedback to keep an Actor resident, suspend it during a longer inference wait, or resume it as the response approaches completion.
Perspective
The article addresses engineers building agent platforms, coding assistants or RL environments on Kubernetes and describes an open-source runtime and how it works with Dynamo and KAI Scheduler; GPU support is being rebuilt on the DRA foundation and parts of the Dynamo integration are experimental.
Agent Substrate is still evolving, its APIs are still changing, GPU support is being rebuilt on the new DRA foundation, and parts of the Dynamo agent integration are experimental; a GPU checkpoint can usually be restored only onto the same GPU model, and sometimes only with the same driver version.