Skip to main content
Back to timeline
arXivSource publication:

Multi-robot planning hands structure to a solver: NSDL reaches 97.0% out-of-distribution accuracy on VIKI-L2

Synopsis

Multi-robot task planning has mostly let an end-to-end vision-language model learn both scene judgment and action structure, and out-of-distribution performance can collapse. NSDL splits the labor: a trained small model decides which goals are already met, how the task splits and which robot does what, and writes these as a plan sketch; a deterministic solver expands the sketch into an executable joint plan. On the public VIKI-L2 benchmark the 7B model reaches 97.0% out-of-distribution accuracy, and three backbones from 3B to 8B all outperform end-to-end and formalize-then-plan baselines.

Source-provided article image: NSDL: Neuro-Symbolic Division of Labor for Multi-Robot Task Planning
arXiv

Interpretation

When a multi-robot team plans from one scene image and one instruction, decisions about which goals are already met, how the task splits and which robot takes which step go to a trained vision-language model; prerequisite actions, handoffs between robots and timing go to a deterministic solver. Earlier end-to-end planners learned both kinds of decision in the model weights, while formalize-then-plan methods had an untrained model write the input of a classical planner. On the public VIKI-L2 benchmark the 7B NSDL model reaches 97.0% out-of-distribution accuracy, against 33.3% for the same-scale end-to-end VIKI-R 7B; the 3B and Qwen3-VL-8B backbones show the same pattern.

The model writes its judgments as a plan sketch that lists the goals still needing action and the intended operations in order, each assigned to a robot; the solver reads only the sketch, not the image and not the goal verdicts. The solver expands in three stages: it adds prerequisite actions and cross-robot handoffs, decomposes each operation into primitive actions, and schedules them under precedence, one action per robot per step, end-effector capacity and effect visibility. On the self-built LOCUS-Bench the runtime rejection rate falls to 1 of 560 rows for NSDL-SFT and 0 of 3018 on VIKI; the same-corpus variant without the solver rises to 23.2% rejections and loses 15.7 points on the equal-weight mean.

Once the solver writes the structure, ordering and scheduling errors nearly disappear, and the remaining failures come from the model misjudging scene states. Of 560 LOCUS rows, 109 are simulator-infeasible, all from solver-accepted schedules, reflecting states, entities or assignments the model wrote wrongly in the sketch. Of the ground-truth-active goals the model misses, 91% are goals it never lists at all; replacing detected relations with ground truth does not reduce the number of never-listed goals.

Perspective

Evaluation is limited to symbolic-replay scoring on the public VIKI-L2 and the self-built LOCUS-Bench, with an action model of the eight primitives of the official VIKI-R implementation, and does not yet cover physical execution. The intended readers are researchers in embodied multi-agent planning and vision-language model training; the stated next steps are teaching the model which goals to list and extending NSDL to physical execution with closed-loop feedback and a larger action model.

Per-cell accuracy on LOCUS is not monotone along either difficulty scale, since family composition differs and deepening structural difficulty also raises assignment difficulty, so the benchmark is not read as a validated two-dimensional capability scale; detector candidate-class macro recall along the structural scale is 11 points below in-distribution, so differences along that scale contain a perception share. Ground-truth relations help only in the relation forms seen in training and are misread in unseen forms; reinforcement learning gains only at the 7B tier, while both 3B checkpoints degrade.