Mine Odyssey benchmarks spatial agents on 180 tasks across 30 real-world Minecraft reconstructions, with GPT-6 Astra at 85.6% success and the strongest open-weight model at 23.9%
The authors introduce Mine Odyssey, a benchmark built from 30 BuildTheEarth Minecraft reconstructions of real-world locations (20 outdoor, 10 indoor, spanning 20 countries and regions on five continents), with 1,059 annotated waypoints and 180 natural-language tasks requiring ordered visits to multiple destinations, paired with a programmable action interface, independent arrival verification, and sandboxed isolation; across eight frontier models, GPT-6 Astra reaches 85.6% success, Claude Opus 5.5 reaches 73.9%, and the strongest open-weight model, DeepSeek-V4.1-Flash, reaches 23.9%, with indoor success generally lower than outdoor.