Skip to main content
Search

# Embodied First Exam

1 related articles

Subscribe to updates
  1. arXiv

    Coding agents take their first physical exam: the best setup solves only 34% of 456 core tasks under real-robot-like conditions

    The Embodied First Exam connects general coding agents to 47 simulated robots and lets each agent choose direct control, code to control, or training a policy, while the real-robot-like Standard Setting supplies only sensor observations and, by default, no resets. Among eight agents, GPT-6 Astra in Codex solves 34% of the Standard tests on 456 core tasks, while on the same 391 tasks it reaches 77% in the Privileged Setting, which exposes the full simulator state and allows unlimited resets. The exam also labels the stage at which failed attempts go wrong, so failures can be compared across settings and agents.

You’re all caught up