Autonomous driving planners stop learning from one score: reflective guidance folded into a single pass, training time cut by over 90%
Synopsis
Autonomous-driving vision-language-action models have learned from a scalar trajectory score that flags a bad plan but cannot say whether scene understanding, behavior selection, or trajectory execution went wrong. Reflect-VLA treats reflective guidance, which pairs evaluation diagnoses with corrective suggestions, as training-time privileged information and internalizes it through dual-path on-policy self-distillation: risky trajectories align with the teacher's token-level corrective distribution, while reliable ones are preserved by self-imitation. It reaches competitive planning performance on NAVSIM v1/v2 and Bench2Drive, cuts training time by over 90% versus reinforcement learning, and generates trajectories in a single pass at inference without guidance or iterative refinement.
Interpretation
A driving planner can now emit a corrected trajectory in one forward pass, with no evaluation feedback at inference. Earlier reinforcement-learning routes used only a scalar trajectory reward, which measures overall quality but cannot separate errors in scene understanding, behavior selection, and trajectory execution; in long-tail scenarios those heterogeneous failures collapse into one number. On NAVSIM v1 the 0.8B, 2B, and 4B models reach PDMS of 89.3, 90.0, and 91.5, beating their corresponding reinforcement-learning baselines by 0.5, 0.8, and 0.3 points.
The corrective signal comes from the same model under a different input condition, not from an extra reward model or manual annotation. Reflective guidance is assembled by a deterministic template from trajectory-evaluation results, geometric differences from a reference trajectory, and the navigation command, and contains a trajectory verdict, planning diagnosis, expert alignment, and refinement directive, so no additional vision-language model is introduced. One checkpoint acts as the student under the base input and as the frozen teacher when it also receives reflective guidance; the student rolls out trajectories, a planning score routes them at a threshold of 0.85, risky trajectories are aligned to the teacher by token-level reverse KL, and reliable ones are kept by self-imitation.
The teacher is not an off-the-shelf model but is built progressively around the mistakes the student actually makes. Offline initial trajectories may not cover the failure patterns that appear during the model's own rollouts, leaving a distribution gap between teacher and student. The teacher is built through driving-knowledge pretraining, planning-and-refinement supervised fine-tuning, and self-refinement rejection sampling, which retains only evaluator-verified refinements that beat their initial trajectories; ablations show rejection sampling alone lowers PDMS from 88.1 to 88.4, but combined with on-policy self-distillation raises it from 90.2 to 91.5.
Perspective
The report covers open-loop and closed-loop evaluation on NAVSIM v1/v2 and Bench2Drive, plus open-loop trajectory evaluation on nuScenes, at model scales of 0.8B, 2B, and 4B, trained on 8 NVIDIA H20 GPUs. It is aimed at researchers and engineers working on end-to-end driving planning, vision-language-action post-training, and distillation. The source states no next-step experiment plan.
Reflective guidance depends on trajectory-evaluation results and a reference trajectory, so constructing it requires an initial prediction and its evaluation. The routing threshold of 0.85 is a compromise between correction and preservation; raising it to 0.95 also distills already reliable trajectories and may disturb effective behavior. Distillation support grows from 16 to 64 with gains, while 256 or the full vocabulary yields only marginal returns.