Robots no longer have to learn only on hardware: Gemini 4 Argon turns sketches, video and photos into runnable MuJoCo scenes
Synopsis
Gemini 4 Argon puts physical understanding, code generation and visual-semantic reasoning into one model so robot tasks can be practiced in simulation first. On the Embodied Reasoning (ERQA) benchmark, which tests physical and spatial problems in 3D environments, it scores 87% using pure reasoning, above other frontier models. It can also reconstruct calibrated simulation twins from robot camera feeds, egocentric human video or hand-drawn sketches, and generate closed-loop control and training data from them.
Interpretation
Robot tasks can now be practiced in simulation before running on real hardware, instead of being tried directly on the robot at every step. Previously this kind of work usually handled physical understanding, code generation and visual reasoning separately, with robot control, Real2Sim and physical understanding each standing alone. On the Embodied Reasoning (ERQA) benchmark, which tests physical and spatial problems in 3D environments, Argon scores 87% using pure reasoning, above every other frontier model.
Argon reasons over multi-view video, 3D geometry, contact mechanics and proprioceptive telemetry, then writes executable control code directly. It combines a visual coordinate harness with action primitives, drives the arm through curl commands, and visually verifies after every turn using the wrist camera image. In a zero-shot control example it makes an arm reorient a fallen mug upright and put a crumpled napkin inside; in MuJoCo practice it controls a Boston Dynamics Spot quadruped to step on a foot-pedal trash bin and dispose of soda cans.
The same reasoning turns real footage or a hand-drawn sketch into a calibrated simulation scene and synthesizes closed-loop trajectories. Previously, building a simulation environment from real recordings or sketches usually required manual modeling and item-by-item calibration. From a hand-drawn sketch it builds a GPU-accelerated mujoco_warp scene, synthesizes a 76-DoF whole-body plus five-finger throw trajectory, verifies 6/6 target knockdown in closed-loop physics, and exports a Blender 4.5 Cycles render.
Simulation-generated data can in turn improve generalist vision-language-action models on real hardware. Previously, improving such models typically depended on collecting more real robot data. From a single real tabletop photo, Argon autonomously builds a calibrated MuJoCo twin, defines new tasks and synthesizes a closed-loop motion generator, collecting 4,500+ domain-randomized episodes, raising real-world success rate from 56% to 75% and cutting completion time from 35s to 24s.
Perspective
The report is for researchers and engineers working on robot learning, simulation and embodied intelligence. The demonstrated scope includes pure-reasoning scores on the ERQA benchmark, zero-shot control for quadruped mobile manipulation and dual-arm manipulation, simulation reconstruction from robot feeds and egocentric human video, throw-trajectory generation from a hand-drawn sketch, multi-granularity hindsight labeling, value function prediction, simulation data improving real-world performance, and porting robotics control papers into MuJoCo WebAssembly browser labs. Argon is available through Google AI Studio, the Gemini API and Google Antigravity, with AI Studio and Antigravity for Google AI Ultra subscribers and the API on the paid tier.
Most capabilities are presented as representative prompt demonstrations rather than independent reproductions; the real-world success rate and completion time gains come from one simulation data augmentation experiment whose repeatability is not stated. The 87% on ERQA is a single benchmark result under a pure-reasoning setting and does not directly extrapolate to other physical tasks.