# Spatial Reasoning

Whether multimodal training changes how a model reasons over text alone had been supported mainly by behavior. Across 10 multimodal configurations from 6 model families, this work builds a vision–language direction and removes each hidden state's projection onto it at inference; in Chameleon-7B, BBH navigate drops while boolean expressions stay largely intact, a double dissociation. The effect is direction-specific and layer-localized, and extends to mental-simulation tasks such as mental rotation, paper folding and perspective taking.

Whether multimodal training changes how a model reasons over text alone had been supported mainly by behavior. Across 10 multimodal configurations from 6 model families, this work builds a vision–language direction and removes each hidden state's projection onto it at inference; in Chameleon-7B, BBH navigate drops while boolean expressions stay largely intact, a double dissociation. The effect is direction-specific and layer-localized, and extends to mental-simulation tasks such as mental rotation, paper folding and perspective taking.

Multi-view 3D vision-language models suffer viewpoint-induced semantic drift: the same object and query produce weaker, diffuse or shifted target responses when the camera moves. LieVLM has geometry tokens predict local rotation parameters and channel-wise gates, applies length-preserving rotations to projected visual features within learned three-dimensional subspaces, and gates the identity-relative correction back into the original representation. On ScanRefer, LieVLM-8B reaches 64.5 Acc@0.25 and 57.2 Acc@0.5, 6.9 and 6.3 points above the published VG LLM-8B under the same proposal-refinement protocol.

Multi-view 3D vision-language models suffer viewpoint-induced semantic drift: the same object and query produce weaker, diffuse or shifted target responses when the camera moves. LieVLM has geometry tokens predict local rotation parameters and channel-wise gates, applies length-preserving rotations to projected visual features within learned three-dimensional subspaces, and gates the identity-relative correction back into the original representation. On ScanRefer, LieVLM-8B reaches 64.5 Acc@0.25 and 57.2 Acc@0.5, 6.9 and 6.3 points above the published VG LLM-8B under the same proposal-refinement protocol.

Ai2 released ToyBlocks, an open simulation framework built around a robot arm and tabletop blocks, the first controlled world in its FoundationWorlds program. It automatically generates tasks and demonstrations, letting researchers vary training data, repeat experiments and trace why a robot fails. In physical tests, the low-level skill model ManiFlow, trained only on simulation data, reached 80.0% success in simulation and 81.1% on the real robot, while the combined high-level planner plus low-level executor succeeded in 55.5% of simulated episodes and 50.0% of real-world episodes.

Ai2 released ToyBlocks, an open simulation framework built around a robot arm and tabletop blocks, the first controlled world in its FoundationWorlds program. It automatically generates tasks and demonstrations, letting researchers vary training data, repeat experiments and trace why a robot fails. In physical tests, the low-level skill model ManiFlow, trained only on simulation data, reached 80.0% success in simulation and 81.1% on the real robot, while the combined high-level planner plus low-level executor succeeded in 55.5% of simulated episodes and 50.0% of real-world episodes.

Google DeepMind's Gemini Robotics ER 2, a high-level reasoning model for robots, is now in public preview on the Gemini platform, letting enterprises declare low-level controls such as vision-language-action models and navigation APIs as tools for the model to orchestrate. The model classifies task progress from continuous video and pinpoints critical event frames, with a stated 91.3% accuracy and 0.96-second mean absolute frame distance. It is described as working across embodiments from stationary arms to humanoids, with simulation, evaluation and deployment handled on Google Cloud.

Google DeepMind's Gemini Robotics ER 2, a high-level reasoning model for robots, is now in public preview on the Gemini platform, letting enterprises declare low-level controls such as vision-language-action models and navigation APIs as tools for the model to orchestrate. The model classifies task progress from continuous video and pinpoints critical event frames, with a stated 91.3% accuracy and 0.96-second mean absolute frame distance. It is described as working across embodiments from stationary arms to humanoids, with simulation, evaluation and deployment handled on Google Cloud.

Vision-language models struggle with spatial reasoning because RGB inputs do not directly supply geometric evidence, and existing fixes either add 3D at inference, paying architecture and latency costs, or train on outcome rewards that supervise only the final answer. GPD renders depth, semantic segmentation and bird's-eye-view layout as compact text, routes the question-relevant subset to the teacher alongside the reference answer, and applies a privileged KL only to incorrect trajectories within GRPO, leaving the deployed model RGB-only. On the 4B backbone it reaches 57.1 on VSI-Bench and 37.6 average across MindCube, SPAR-Bench, MMSI-Bench and ViewSpatial, above GRPO and answer-privileged OPSD.

Vision-language models struggle with spatial reasoning because RGB inputs do not directly supply geometric evidence, and existing fixes either add 3D at inference, paying architecture and latency costs, or train on outcome rewards that supervise only the final answer. GPD renders depth, semantic segmentation and bird's-eye-view layout as compact text, routes the question-relevant subset to the teacher alongside the reference answer, and applies a privileged KL only to incorrect trajectories within GRPO, leaving the deployed model RGB-only. On the 4B backbone it reaches 57.1 on VSI-Bench and 37.6 average across MindCube, SPAR-Bench, MMSI-Bench and ViewSpatial, above GRPO and answer-privileged OPSD.

The authors introduce SpatialOPSD, an on-policy self-distillation framework that treats verified execution traces from a spatial coding agent as privileged information, letting a frozen teacher initialized from the same checkpoint see the traces while the student sees only raw images, with Repetition-Aware Distillation (repetition masking plus unlikelihood regularization) suppressing verbatim copying; on MindCube-Tiny and ViewSpatial, the 9B student raises average accuracy from 45.9 to 51.9, above SFT at 47.6 and GRPO at 48.2, and retains about 91% of its tool-using teacher SpatialClaw at 57.3 in a single tool-free forward pass.

The authors introduce SpatialOPSD, an on-policy self-distillation framework that treats verified execution traces from a spatial coding agent as privileged information, letting a frozen teacher initialized from the same checkpoint see the traces while the student sees only raw images, with Repetition-Aware Distillation (repetition masking plus unlikelihood regularization) suppressing verbatim copying; on MindCube-Tiny and ViewSpatial, the 9B student raises average accuracy from 45.9 to 51.9, above SFT at 47.6 and GRPO at 48.2, and retains about 91% of its tool-using teacher SpatialClaw at 57.3 in a single tool-free forward pass.

The work introduces Flow-of-Thought (FoT), a framework that trains coordinate-, time-, and action-conditioned spatial U-Nets with flow matching on rotation orbits and cumulative shortest-path prefixes, then freezes the learned dynamics; at inference, the source and its horizontal reflection serve as competing hypotheses selected by foreground-weighted reconstruction energy. On locked tests FoT reaches 100.0% on Tetris and 99.0% on colored shapes, with maze endpoint IoU 97.5% and 100% goal reach; under frozen transfer to 133 BLINK Multi-view public validation pairs, the orbit-trained flow reaches 72.2% versus 63.9% for its endpoint-only control.

The work introduces Flow-of-Thought (FoT), a framework that trains coordinate-, time-, and action-conditioned spatial U-Nets with flow matching on rotation orbits and cumulative shortest-path prefixes, then freezes the learned dynamics; at inference, the source and its horizontal reflection serve as competing hypotheses selected by foreground-weighted reconstruction energy. On locked tests FoT reaches 100.0% on Tetris and 99.0% on colored shapes, with maze endpoint IoU 97.5% and 100% goal reach; under frozen transfer to 133 BLINK Multi-view public validation pairs, the orbit-trained flow reaches 72.2% versus 63.9% for its endpoint-only control.

Through diagnostic analyses, the work locates the bottleneck of vision-language models' spatial weakness in a missing view-consistent spatial interface rather than insufficient visual features or weak reasoning, and introduces 3ViewSense, a framework that grounds spatial reasoning in orthographic views and uses a Simulate-and-Reason mechanism to align egocentric perception with allocentric references, enabling explicit mental rotation and reconstruction; it significantly outperforms existing baselines on spatial reasoning benchmarks, with consistent gains on occlusion-heavy counting and view-consistent spatial reasoning, and also improves the stability and consistency of spatial descriptions.

Through diagnostic analyses, the work locates the bottleneck of vision-language models' spatial weakness in a missing view-consistent spatial interface rather than insufficient visual features or weak reasoning, and introduces 3ViewSense, a framework that grounds spatial reasoning in orthographic views and uses a Simulate-and-Reason mechanism to align egocentric perception with allocentric references, enabling explicit mental rotation and reconstruction; it significantly outperforms existing baselines on spatial reasoning benchmarks, with consistent gains on occlusion-heavy counting and view-consistent spatial reasoning, and also improves the stability and consistency of spatial descriptions.