Spatial reasoning stops guessing depth: 3D cues only in training, RGB-only at inference, 37.6 average on four benchmarks
Synopsis
Vision-language models struggle with spatial reasoning because RGB inputs do not directly supply geometric evidence, and existing fixes either add 3D at inference, paying architecture and latency costs, or train on outcome rewards that supervise only the final answer. GPD renders depth, semantic segmentation and bird's-eye-view layout as compact text, routes the question-relevant subset to the teacher alongside the reference answer, and applies a privileged KL only to incorrect trajectories within GRPO, leaving the deployed model RGB-only. On the 4B backbone it reaches 57.1 on VSI-Bench and 37.6 average across MindCube, SPAR-Bench, MMSI-Bench and ViewSpatial, above GRPO and answer-privileged OPSD.
Interpretation
Training for spatial reasoning can now carry the scene's true geometry while the deployed model still receives only RGB images and the question. Two prior routes sat at opposite ends: explicit 3D methods integrate geometric encoders or reconstruction tools into the model at the cost of architectural change and extra inference overhead, while training-based methods keep RGB inference but supervise only the final answer, so a wrong depth order or flipped left-right relation along the reasoning chain is neither localized nor corrected. On Qwen3-VL-4B, GPD reaches 57.1 on VSI-Bench, 1.7 points above the base model, 0.7 above GRPO and 1.4 above OPSD; across MindCube, SPAR-Bench, MMSI-Bench and ViewSpatial it averages 37.6, 1.9 points above GRPO and 6.1 above OPSD.
Geometric cues enter the teacher as text, and only the part relevant to the current question is selected. Injecting all 3D information indiscriminately can introduce irrelevant context and dilute the teacher signal, so evidence must be chosen per question type. Offline, fixed templates convert depth, semantics and bird's-eye-view layout into text summaries, which the base model then routes by question; ablations give routed text an average of 41.5, above full 3D text injection at 39.8 and routed images at 40.4.
Distillation acts only on trajectories the student got wrong, leaving correct ones untouched. Applying distillation to all trajectories is suboptimal, because when the student already answers correctly the teacher signal may add noise or conflicting gradients. Applying the privileged KL to all trajectories rather than incorrect ones alone lowers the average by 0.9 points to 40.6; among incorrect-only variants the average rises from 39.6 at a low weight to 41.5 at the chosen weight, then falls to 40.2 and 39.6 as the weight grows.
Perspective
The report covers Qwen3-VL-Instruct backbones at 2B and 4B, trained on a mixture of 10K VSI, 4K SPAR and 1K MindCube training-split samples, and evaluated on five spatial reasoning benchmarks: VSI-Bench, MindCube, SPAR-Bench, MMSI-Bench and ViewSpatial. It suits researchers working on spatial reasoning in vision-language models and reinforcement-learning post-training.
Ablations show that too small a privileged-KL weight gives insufficient process-level correction while too large a weight over-constrains the policy and degrades performance, so the weight has an effective range. In the internalization analysis, GPD gains less than GRPO when routed 3D text is injected at test time, which the authors read as the geometric cues having been internalized during training rather than remaining a test-time dependency.