Ai2 releases ToyBlocks, a tabletop block world that makes robot learning reproducible, with simulated and real skill success rates close

Robot learning gains a deliberately narrow controlled simulation world where researchers can change training data or model design, repeat the same test and trace which stage a failure came from. Robot demonstrations have often been narrow and do not explain what a robot learned or why it fails; open-ended domains model so many objects and interactions that realistic physics, automatic generation at scale, reasoning combined with dexterity, and broad task coverage cannot all be met at once. ToyBlocks' narrow scope lets it meet all four requirements, and it pairs automatic task and demonstration generation with metrics that measure robot behavior over time and with high- and low-level oracles that isolate whether planning or execution failed.

Training data no longer depends on human teleoperation but is generated automatically in simulation with labels. Teleoperation, in which a person remotely controls the robot, is time-consuming and hard to scale; ToyBlocks instead generates short-horizon skills with reinforcement learning and motion planning in Isaac Lab, then has a task-and-motion planner select and sequence them. Generating and executing hundreds of plans in parallel on a single GPU reaches up to 24 times real-time performance; the initial data release contains 85,336 long-horizon demonstrations, nearly 1,600 hours of robot execution data and 144 million frames.

Policies trained in simulation transfer zero-shot to a physical robot, and simulated skill success rates are close to real ones. It was unclear whether behaviors and failure patterns seen in simulation indicate real-world performance, which requires comparing the same models in both settings. ManiFlow's atomic-skill success rates were 80.0% in simulation and 81.1% in reality; the hierarchical Molmo 2 plus ManiFlow system succeeded in 55.5% of simulated episodes (111/200) and 50.0% of real-world episodes (15/30) across thirty scene configurations.

The demonstrated scope is simulated tabletop arm-and-block manipulation, including stacking, unstacking, flipping, rotating and spelling tasks, plus zero-shot real-robot tests on a Franka FR3 arm with a Robotiq gripper across thirty scene configurations. The authors say they target a second release in a few weeks with a larger-scale dataset, more skills and more comprehensive benchmarking, and will add fully integrated reinforcement-learning training not contained in the current package.

The authors state that further trials are needed to establish statistical confidence across tasks for the hierarchical system's real-world results. In the agentic spelling experiment, the agent solved or correctly declared infeasible 63% of tasks, with errors mostly from confusing look-alike letters (13%) or choosing an incorrect letter orientation (15%); the authors also note that current frontier models often make similar perceptual errors. In the world-model tests, 900 trajectories were generated per skill, half successful demonstrations and half unsuccessful because of small changes to spatial configurations.