AgentGarten splits an environment in two: a simulator or game engine keeps persistent state and executes programmable interaction rules, while a shared neural renderer turns the geometry conditions the engine exports into the observations an agent sees, in real time. The renderer comes from adapting a pretrained video model to geometry conditions, converting it to block-causal generation, and distilling it with Adversarial Forcing, generating observations above 35 frames per second on one H100. In hide-and-seek, pretrained agents that perceive only rendered frames built shelter with panels by round 4 and crossed walls with a ramp by round 10, and four further worlds improved over four rounds.
AgentGarten splits an environment in two: a simulator or game engine keeps persistent state and executes programmable interaction rules, while a shared neural renderer turns the geometry conditions the engine exports into the observations an agent sees, in real time. The renderer comes from adapting a pretrained video model to geometry conditions, converting it to block-causal generation, and distilling it with Adversarial Forcing, generating observations above 35 frames per second on one H100. In hide-and-seek, pretrained agents that perceive only rendered frames built shelter with panels by round 4 and crossed walls with a ramp by round 10, and four further worlds improved over four rounds.