Robots get a cloud brain: Gemini Robotics ER 2 enters public preview with 91.3% key-frame accuracy
Synopsis
Google DeepMind's Gemini Robotics ER 2, a high-level reasoning model for robots, is now in public preview on the Gemini platform, letting enterprises declare low-level controls such as vision-language-action models and navigation APIs as tools for the model to orchestrate. The model classifies task progress from continuous video and pinpoints critical event frames, with a stated 91.3% accuracy and 0.96-second mean absolute frame distance. It is described as working across embodiments from stationary arms to humanoids, with simulation, evaluation and deployment handled on Google Cloud.
Interpretation
Instead of executing only single-step commands, a robot can use a high-level reasoning model to judge from continuous video whether each step succeeded and adjust later steps when it did not. The prior approach broke tasks into preset scripts that robots followed step by step, making it hard to reroute around unanticipated situations. The model classifies task progress into binary status, discrete categories or continuous scores, which the announcement says supports real-time adjustment or retry of failed steps without restarting the whole workflow.
The model drives robots by having developers declare low-level control interfaces as tools, including vision-language-action models, navigation APIs or custom functions, with video, audio and text streamed in as input. The vision-language-action model handles concrete motion while the high-level reasoning decides what to do next, and the two connect through tool calls. The stated key-frame metric is 91.3% accuracy with a 0.96-second mean absolute frame distance, and the announcement says it runs at 4x the execution speed and a fraction of the compute cost of many larger model categories.
One reasoning layer orchestrates robots across embodiments and supports multi-robot division of labor and task handoff. Robots of different forms previously ran separate control stacks and could not easily share a semantic understanding of the workspace. The stated coverage runs from stationary arms and wheeled rovers to quadrupeds and more complex forms, and the announcement says different robot types can share semantic understanding, divide labor and hand off tasks.
Perspective
The public material describes Gemini Robotics ER 2 in public preview on the Gemini platform, aimed at developers and enterprise teams building embodied agents in simulation, real robot control or teleoperation; the reported figures cover the key-frame localization capability.
The 91.3% accuracy and 0.96-second mean absolute frame distance are official figures for a single capability, and the material does not state the evaluation dataset or task distribution; cross-embodiment generalization and multi-robot collaboration have no corresponding quantitative results.