Bone hyperspectral 3D reconstruction drops external camera poses: BoNeRF-HS tops PSNR and SSIM on mouse bone and two bone analogues
Synopsis
Low texture, specular reflection and irregular morphology make external pose estimation such as COLMAP unreliable on bone surfaces, yet existing hyperspectral 3D reconstruction methods depend on such poses. BoNeRF-HS jointly optimizes camera intrinsics, per-view poses, volume density and the hyperspectral radiance field, and adds a gated spectral adapter head that learns a residual correction across neighbouring bands. On 45-view, 120-band, 400–1000 nm data from mouse bone, a trabecular bone analogue and a cortical bone analogue, it gives the highest PSNR and SSIM on all three specimens, with PSNR 29.94 and SSIM 0.9254 on mouse bone.
Interpretation
Hyperspectral 3D reconstruction can proceed without externally supplied camera poses, because camera intrinsics and each view's pose are learned together with the radiance field from multi-view hyperspectral images. Earlier hyperspectral neural rendering methods assume camera parameters are known or externally estimated, while the low texture and specular reflection of bone surfaces make feature-based structure-from-motion pipelines such as COLMAP prone to unreliable pose estimation. Across the three specimens, BoNeRF-HS reaches higher PSNR and SSIM than NeRF–24, SiNeRF and NoPe-NeRF, all converted to hyperspectral output; on mouse bone it gives PSNR 29.94, SSIM 0.9254 and RMSE 0.0325, and on the cortical bone analogue PSNR 31.78, SSIM 0.9232 and RMSE 0.0258.
Spectral prediction is split into a direct estimate and a gated residual correction, so correlation between neighbouring bands can be exploited separately. A direct spectral head treats the bands as independent output channels and does not explicitly model correlation between contiguous bands. In an ablation averaged over the three specimens, the adapter with a learned gate raises PSNR from 26.32 for the direct spectral head to 29.29, SSIM from 0.8428 to 0.8810 and lowers RMSE from 0.0500 to 0.0376, while the fixed weak gate variant falls below the direct head.
Perspective
The experiments cover one mouse bone, one open-cell PCF15 trabecular bone analogue and one closed-cell PCF40 cortical bone analogue, each with 45 views and 120 bands over 400–1000 nm, with about 90% of views used for training and the rest for testing; acquisition used a fixed camera, fixed illumination and a motorised turntable in a benchtop setting. The authors state that translation to real surgical use will require validation under less controlled acquisition conditions.
The authors state that each type has a single sample, so the reported improvements should be read as promising initial evidence rather than a statistically validated benefit; the motivation that bone surfaces defeat COLMAP is not empirically demonstrated on this dataset by showing COLMAP failing or producing high pose error; and on the soft-tissue sample BoNeRF-HS leads on PSNR and RMSE but has slightly lower SSIM than NoPe-NeRF (0.8373 versus 0.8472), which the authors flag as an open question.