A verifiable proving ground for egocentric AI assistants: humans act on instructions, and the best model tops out at 20% on long-horizon guidance
Synopsis
Testing egocentric AI assistants has usually required real people, and the expensive, scarce interaction data left open whether today's multimodal models can actually help someone in the physical world. InteractiveWorld moves that loop into a verifiable environment: the assistant issues instructions, a human agent observes the scene, understands the instruction and acts independently, and the environment executes the action and updates what both see. InteractiveBench built on it holds 120 multi-turn tasks across six categories, each instruction shaping the outcome and scored by verifiable state-based rewards; across 12 frontier multimodal models, Claude Fable 5.1 reaches at most 20% and 25% on long-horizon Guidance and Planning tasks, respectively.
Interpretation
Evaluation of egocentric AI assistants now has a verifiable closed loop in which the assistant does not drive a robot directly but instructs a human agent that observes, decides and acts on its own. Earlier testing usually needed real people, and real interaction data was expensive and rare, leaving unanswered whether today's models can truly help people in the physical world. The environment is described as the first verifiable environment for closed-loop interaction between a human and an AI assistant, distinguished from robot-centric simulators.
The loop runs in three steps: the assistant gives an instruction, the human agent acts on it, and the environment executes the action and updates what is seen next. This division confines the assistant to the role of guide while the human agent independently observes, understands and manipulates, reproducing the instruction-and-action relation of real collaboration. InteractiveBench built on this environment contains 120 multi-turn tasks across six categories, with the assistant guiding in real time and every instruction shaping the outcome.
Task outcomes are scored by verifiable state-based rewards rather than human judgment. Verifiable rewards let the success or failure of a multi-turn interaction be checked automatically, in contrast with testing that requires real people. Twelve frontier multimodal models were tested on the benchmark, and the results show they remain far from reliable assistants.
The strongest model's scores on long-horizon tasks expose where current assistants fall short. Even the best performer, Claude Fable 5.1, reaches at most 20% and 25% on long-horizon Guidance and Planning tasks, respectively. This comes from experiments with 12 frontier multimodal models across the six task categories of InteractiveBench.
Perspective
The demonstrated scope is a closed loop in InteractiveWorld where a human agent performs the actions and the assistant only guides, scored on 120 multi-turn tasks in six categories of InteractiveBench with verifiable state-based rewards across 12 frontier multimodal models. It is aimed at researchers and engineers working on egocentric assistants and interactive multimodal agent evaluation.
The material calls the environment the first verifiable environment for closed-loop human-AI interaction and InteractiveBench the first dynamic benchmark for egocentric AI assistants, and these first-of-their-kind statements come from the work itself. The strongest model reaches at most 20% and 25% on long-horizon Guidance and Planning tasks, respectively, and the material reads this as showing current models are still far from reliable assistants.