Building inspection stops at pixels: pulling Gaussian centers onto real surfaces cuts height error to 0.33%
Lead
Three-dimensional Gaussian reconstruction for building inspection no longer aims only at photorealistic views: with region and point-wise depth supervision, Gaussian centers are pulled toward physical surfaces, cutting depth error by 58.1% and 86.8% on synthetic UAV and smartphone-captured data, with a mean relative height-measurement error of 0.33%.
Story
The 3D representation used for building inspection is no longer only something that looks right; it can be used to measure component dimensions and areas directly. Image-based reconstruction already offered a scalable representation, but photorealistic reconstruction alone did not fully provide the geometric measurability, semantic structure, and interactive information access that engineering applications require. On the synthetic UAV-inspection QuakeCity dataset and the smartphone-captured HKUPB dataset, Gaussian-center depth MAE drops by 58.1% and 86.8%.
The method adds region constraints, depth constraints, and class features to each Gaussian point, so optimization keeps it on the target component and on the real surface. Mask-based region supervision suppresses irrelevant backgrounds and focuses optimization on target regions; point-wise depth supervision uses multi-view stereo point clouds as reference geometry to guide Gaussian centers toward physical surfaces, providing geometric anchors for measuring dimensions and areas. A semantic Gaussian field associates learnable class features with individual Gaussian primitives, reaching mIoU of 0.8529 and 0.8642 on QuakeCity and HKUPB; on HKUPB the mean relative errors for height, width, and area measurements are 0.33%, 1.07%, and 0.60%.
These capabilities are packaged into an interactive inspection platform supporting real-time visualization, semantic analysis, quantitative measurement, and multimodal large language model-assisted interaction. The platform, AEC-SplatTwin, turns the reconstruction and semantic capabilities from offline outputs into an operational way to access information during inspection. The platform runs on the synthetic UAV-inspection QuakeCity and smartphone-captured HKUPB datasets, covering real-time visualization, semantic analysis, quantitative measurement, and multimodal large language model-assisted interaction.
What to watch
This report draws on summary-level material covering depth error, semantic mIoU, and dimensional measurement error on the synthetic UAV-inspection QuakeCity and smartphone-captured HKUPB datasets, plus the AEC-SplatTwin platform's real-time visualization, semantic analysis, quantitative measurement, and multimodal large language model-assisted interaction. It is intended for researchers and engineers following 3D reconstruction and building inspection who want to know what the framework does.
The material is a summary and does not give the full network architecture, training details, or error-statistics setup; depth supervision depends on multi-view stereo point clouds as reference geometry, and the material does not state how their quality affects results; measurement errors are reported only on HKUPB, so performance in other settings remains unclear.