Spatial reasoning moves past what is visible: 3,862 questions ask what an intervention changes, and the best model reaches 58.0% against 87.2% for humans

The target of spatial reasoning evaluation changes: instead of asking about relations already visible in the frames, it asks what those relations become in a scene state that follows an intervention and is never observed. Earlier spatial benchmarks center on spatial perception, reading off relations present in the observed frames, and do not require reasoning about how relations change after an intervention. The benchmark holds 3,862 questions from 182 real scenes across 16 task types, organized into static perception, local prediction and global prediction, and covering direction, distance, visibility and occlusion, and attachment structure.

A model must first reconstruct the scene from multi-view observations, then apply the transformation, and finally answer the query in the unobserved result state. This observe–transform–infer pipeline separates predictive spatial reasoning from the perceptual task of reading off relations already visible in the input. Questions come from a geometry-grounded pipeline that builds depth-verified view chains from posed RGB-D scans, applies controlled local or global transformations in 3D, and derives programmatic labels while keeping the resulting state unobserved.

Across 21 evaluated models, a clear gap opens between the strongest model and human performance, while spatially specialized models stay near random chance. The strongest model, Gemini 3.5 Flash, reaches 58.0% against 87.2% for humans and a 25.9% random baseline, while spatially specialized models range from 25.7% to 29.1%. The evaluation covers 9 proprietary, 6 general-purpose open-weight and 6 spatially specialized models under a chain-of-thought prompt with deterministic decoding, scored by the macro-average of per-task accuracy across the 16 task types.

Post-training on data from the same pipeline lifts a 4B model from 34.0% to 65.7% overall on the benchmark and improves the macro average across six out-of-domain benchmarks. Qwen3-VL-4B starts at 34.0%, staged SFT reaches 65.7%, and the out-of-domain macro average rises from 33.5% to 38.1%. The training set is generated by the same programmatic pipeline from scenes disjoint from the benchmark, with 12,249 questions covering all 16 task types, and compares mixed SFT, staged SFT and GRPO.

The benchmark targets indoor scenes and draws its questions from the validation splits of ScanNet and ScanNet++, with ScanNet++ contributing 3,144 questions; it evaluates vision-language models that see only pre-transformation views. A reader interested in outdoor settings or in object arrangements and scales absent from these two sources would need separate validation.

Question labels depend on reconstructed meshes, depth maps, camera poses and instance-level semantic annotations, so geometric errors can affect ray-cast visibility, apparent surface contact or support relationships, and whether a queried object is sufficiently recognizable in a selected view; reconstruction-related uncertainty in occlusion and attachment labels cannot be fully eliminated, particularly for objects with incomplete geometry or low reconstruction confidence. The three intervention types update scene geometry programmatically without simulating physical dynamics, so the benchmark does not assess deformation, friction or collision-driven motion.