One fringe image, one height map: a hybrid Mamba-Transformer network reaches 0.548 mm RMSE
Synopsis
Reconstructing a 3D surface from a single high-frequency fringe image is hard because long-range spatial consistency and fine geometric detail must both survive. Convolutional networks model long-range context weakly, and window-based Transformers build global interactions only gradually with depth. A hybrid Mamba-Transformer network adds visual state-space blocks at the intermediate-resolution encoder stage of a Swin-style U-shaped architecture, with an axial-dilated block that strengthens directional fringe features before selective scanning. On a public fringe-projection dataset, using only one deformed fringe image, it reaches 0.548 mm RMSE and 0.9985 SSIM, cutting RMSE by 65.2% and 59.6% relative to UHRNet and HiDNet.
Interpretation
A single deformed fringe image can be regressed directly to a surface height map in millimetres, without multistep phase shifting or temporal phase unwrapping. Multistep phase shifting and temporal unwrapping need several sequential fringe frames, which are prone to motion-induced error and poorly suited to dynamic scenes; traditional single-shot methods such as Fourier transform profilometry suffer from spectral leakage, frequency aliasing and phase errors. Under a unified training and evaluation protocol built on public fringe-projection data, the network reaches 0.195 mm MAE, 0.548 mm RMSE, 0.043 AbsRel and 0.9985 SSIM, better than hNet, SwinConvUNet, UHRNet and HiDNet on all four metrics.
The network places long-range modelling at the intermediate-resolution encoder stage, letting selective state-space scanning carry image-wide consistency while window attention handles local detail. An all-Swin U-shaped baseline uses Swin Transformer blocks throughout the L1 stage; replacing that stage with visual state-space blocks lowers MAE from 0.369 to 0.215 mm and RMSE from 1.089 to 0.643 mm. Ablations run on 3,070 fringe-depth pairs, with 74 samples held out as an independent test set excluded from parameter optimization and model selection; extending state-space blocks to the highest-resolution L0 stage brings no further gain.
The axial-dilated visual state-space block rewrites input features with parallel local, dilated and axial depth-wise convolutions before four-directional selective scanning. Compared with a configuration using only standard visual state-space blocks, the complete block lowers MAE further from 0.215 to 0.195 mm and RMSE from 0.643 to 0.548 mm. Removing the axial branch raises RMSE to 0.580 mm, and removing the post-scan local residual injection raises it to 0.608 mm; the complete configuration is best on all four metrics.
Perspective
The demonstrated scope is single-channel input from the ninth frame of a public fringe-projection dataset at 768×512 pixels with 128 vertical fringes, evaluated on 74 samples fixed out of 3,070 and excluded from training and model selection; reconstructions are also shown for one object, two spatially isolated objects and several partially overlapping objects.
Generalization across different cameras, projection systems, working distances, fringe densities and object materials remains to be established; highly reflective or non-Lambertian surfaces, shadows, steep slopes and low-modulation regions may still produce locally elevated errors. Five-channel input improves all metrics for the all-Swin architecture but degrades both selective-scan architectures, which the authors attribute to a possible mismatch between simple channel concatenation and the current selective-scan feature embedding and say needs further experiments to establish.