A common arena for long-horizon multi-robot tasks: top models fail many clean episodes, and online perturbations cut success further

Long-horizon embodied agents now have a single arena that covers both task execution unfolding over time and coordination among robots with different capabilities. Existing benchmarks investigate dynamic task execution and embodied multi-agent coordination separately, and provide limited evaluation of their intersection. The evaluation covers locally deployable open-weight planners, multi-agent state-tracking strategies, and frontier models, across increasing task horizons.

Agents perform only task-level reasoning, while perception and low-level control are handed to a structured skill-level interface. That skill-level interface separates task-level reasoning from perception and low-level control, so the evaluation is architecture-agnostic. Evaluation runs under controlled online perturbations, including execution failures, task interruptions, and mission updates, in scenarios requiring coordination across robots with heterogeneous capabilities.

Supervised adaptation greatly improves a small model, yet performance remains substantially lower on longer and multi-robot missions. Even the strongest evaluated models fail on a substantial fraction of clean episodes, and their success rates decrease further under online perturbations. Released with the benchmark are 261,646 task-level training examples and an architecture-agnostic expert-trace generation pipeline.

The benchmark targets persistent task reasoning in dynamic multi-robot environments, evaluating locally deployable open-weight planners, multi-agent state-tracking strategies, and frontier models across increasing task horizons under controlled online perturbations such as execution failures, task interruptions, and mission updates. Released with it are 261,646 task-level training examples and an architecture-agnostic expert-trace generation pipeline.

The material does not state how perturbation strength maps onto task horizon, nor the size of the data used for supervised adaptation; the adapted small model improves but remains substantially lower on longer and multi-robot missions.