SLA moves spatial alignment into continual pre-training: 83.8% average success across eight real deployment conditions, 24.0 points over the baseline
Synopsis
SLA extends spatial alignment for vision-language-action models from task post-training into continual pre-training, and keeps it during downstream post-training. Initialized from a pretrained policy, it jointly optimizes action prediction and a cosine alignment between intermediate visual embeddings and spatial representations from a frozen 3D perception model, using feature-dependent gates to aggregate multiple VLA layers and probabilistic state dropout to reduce reliance on demonstration trajectories. It reaches 72.3% on the RoboTwin 2.0 Clean2Random setting and 83.8% average success across eight real deployment conditions, 24.0 percentage points above the baseline.
Interpretation
Spatial understanding for vision-language-action models no longer has to be added only during task post-training; SLA moves it into continual pre-training and keeps it during downstream post-training. The earlier spatial-alignment approach, Spatial Forcing, applied spatial supervision only during post-training, leaving spatial competence closer to a task-specific add-on. On all 50 RoboTwin 2.0 tasks, with alignment in both stages, Clean2Random success rises from 49.2% at 5% of the data to 72.3% at full data; post-training-only alignment gives 57.1% and continual-pre-training-only alignment gives 63.2%.
During training, intermediate visual embeddings are aligned to spatial representations from a frozen 3D perception model, without changing the action backbone or the action head. The alignment signal comes from pixel-level 3D representations produced by OmniVGGT, which can use RGB, camera intrinsics and extrinsics, and depth maps together, filling in placeholder tokens when annotations are missing. Changing only the spatial supervision source during post-training raises average success on RoboTwin clean demonstrations from 31.8% with no alignment to 42.8% with an RGB-only OmniVGGT teacher and 46.0% when camera calibration is supplied.
Visual embeddings from multiple layers are adaptively weighted by feature-dependent gates before receiving spatial supervision, avoiding a hand-picked fixed layer. A fixed layer requires separate training experiments, and a layer suited to one data domain need not suit another. On the 18-layer PaliGemma language backbone, single-layer alignment gives Clean2Random success between 43.4% and 67.3%, with layer 12 best; adaptive fusion reaches 72.3% and 79.2% average, 5.0 and 3.5 points above layer 12.
The entire robot-state token segment is randomly dropped during training, pushing the policy toward visual cues. Robot states can become a shortcut, letting a policy associate particular states with demonstrated trajectories instead of responding to visual changes in the scene. On all 50 RoboTwin tasks, a 25% state dropout probability yields 79.2% average success versus 38.1% with no dropout; the next-best setting, 75%, reaches only 61.9%.
Perspective
The demonstrated scope is three simulation benchmarks, one real-world benchmark and eight private real deployment conditions, on a bimanual Piper platform and a single-arm Franka robot with Realsense D435 cameras; the policies on RoboDojo-Real and RoboDojo-Sim additionally use a memory mechanism during post-training, and memory frames are excluded from the spatial alignment branch.
The source reports that the response to state dropout is not monotonic and that removing state inputs from every training sample is also suboptimal; per-task RoboTwin results show clear gaps between clean and randomized conditions, such as Place Cans Plasticbox falling from 96% to 40% and Handover Block from 91% to 38%; on RoboDojo-Sim randomized layouts substantially reduce success, and the Long-Horizon dimension trails most baselines.