Loan Bernat

MAGMA-Bench evaluates long-horizon embodied agents in dynamic multi-robot environments, adding controlled online perturbations such as execution failures, task interruptions and mission updates, and using a skill-level interface to separate task-level reasoning from perception and low-level control. Earlier benchmarks studied dynamic execution or multi-robot coordination largely one at a time. Even the strongest evaluated models fail on a substantial fraction of clean episodes, success drops further under perturbation, and supervised adaptation greatly improves a small model yet stays substantially lower on longer and multi-robot missions.

MAGMA-Bench evaluates long-horizon embodied agents in dynamic multi-robot environments, adding controlled online perturbations such as execution failures, task interruptions and mission updates, and using a skill-level interface to separate task-level reasoning from perception and low-level control. Earlier benchmarks studied dynamic execution or multi-robot coordination largely one at a time. Even the strongest evaluated models fail on a substantial fraction of clean episodes, success drops further under perturbation, and supervised adaptation greatly improves a small model yet stays substantially lower on longer and multi-robot missions.