Qinsi Wang

Testing egocentric AI assistants has usually required real people, and the expensive, scarce interaction data left open whether today's multimodal models can actually help someone in the physical world. InteractiveWorld moves that loop into a verifiable environment: the assistant issues instructions, a human agent observes the scene, understands the instruction and acts independently, and the environment executes the action and updates what both see. InteractiveBench built on it holds 120 multi-turn tasks across six categories, each instruction shaping the outcome and scored by verifiable state-based rewards; across 12 frontier multimodal models, Claude Fable 5.1 reaches at most 20% and 25% on long-horizon Guidance and Planning tasks, respectively.

Testing egocentric AI assistants has usually required real people, and the expensive, scarce interaction data left open whether today's multimodal models can actually help someone in the physical world. InteractiveWorld moves that loop into a verifiable environment: the assistant issues instructions, a human agent observes the scene, understands the instruction and acts independently, and the environment executes the action and updates what both see. InteractiveBench built on it holds 120 multi-turn tasks across six categories, each instruction shaping the outcome and scored by verifiable state-based rewards; across 12 frontier multimodal models, Claude Fable 5.1 reaches at most 20% and 25% on long-horizon Guidance and Planning tasks, respectively.