Skip to main content
Back to timeline
arXivSource publication:

Depth boundaries are a sliver of pixels but nearly half the error: E3Stereo cuts IGEV boundary error from 3.77 to 3.23 pixels

Synopsis

Stereo matching error concentrates at depth boundaries: on IGEV, geometric edge pixels are a small fraction of evaluated pixels yet contribute about half of total absolute disparity error. E3Stereo learns geometric edges from disparity supervision and uses the same edge map both to guide full-resolution disparity residual correction and to weight boundary errors during training, jointly optimized with the stereo backbone. On Scene Flow it reduces boundary end-point error by 13.4%, 14.3% and 9.5% for RAFT-Stereo, IGEV and MonSter while lowering overall error, but direct transfer to real datasets is worse than IGEV and only target-domain fine-tuning brings gains across all reported ETH3D and KITTI metrics.

Source-provided article image: E3tereo: Improving Stereo Matching at Depth Boundaries with Learned Geometric Edges
arXiv

Interpretation

Stereo matching error is heavily concentrated at depth boundaries, and photometric edges are not the same as depth discontinuities, so refining boundaries from brightness edges can target the wrong locations. An evaluation of seven existing stereo methods on Scene Flow shows error peaking at depth boundaries and dropping to roughly one quarter of that level just one pixel away. On IGEV, geometric edge pixels are only a small fraction of evaluated pixels but contribute about half of the total absolute disparity error, tying boundary error directly to overall error.

E3Stereo uses one geometric edge map learned under disparity supervision for two jobs: predicting a full-resolution disparity residual and increasing the penalty on boundary pixels during training. Earlier boundary refinement relied on image-guided refinement, joint edge detection, or modeling multiple disparity modes at edge pixels, with edge cues coming from image appearance rather than disparity structure. On the Scene Flow test set, adding the components to RAFT-Stereo, IGEV and MonSter reduces boundary end-point error by 13.4%, 14.3% and 9.5%, and overall end-point error from 0.72 to 0.67, 0.47 to 0.43, and 0.37 to 0.35 pixels, respectively.

The edge predictor is trained jointly with disparity refinement rather than frozen after separate pretraining, so the edge representation stays geometrically supervised while adapting to refinement. A frozen pretrained GeoEdgeNet configuration gives overall end-point error of 0.46 pixels and boundary end-point error of 3.53 pixels, versus 0.43 and 3.23 pixels when jointly trained. The IGEV-based model pretrained on Scene Flow and then fine-tuned beats IGEV on all eleven reported metrics across ETH3D, KITTI 2012 and KITTI 2015, for example ETH3D Bad1.0 (Noc) falling from 1.12% to 0.48%.

Perspective

The demonstrated scope is Scene Flow pretraining and evaluation plus fine-tuned results with IGEV as the reference backbone on ETH3D, KITTI 2012 and KITTI 2015; official Middlebury 2014 test results are not reported because the submission service is disabled. The intended readers are researchers and engineers working on stereo matching and depth estimation, and the method attaches to existing backbones as a plugin.

When transferred directly from Scene Flow to real datasets, E3Stereo has higher error rates than IGEV in all five settings, for example rising from 5.14% to 5.59% on KITTI 2012 and from 3.61% to 3.97% on ETH3D. This contrast is consistent with sensitivity to domain shift but does not isolate the contributions of edge prediction, residual refinement and training weights to the transfer gap. Separately, RCF has higher edge-detection F1 than Canny yet larger boundary disparity error, indicating edge-detection quality alone does not determine guidance effectiveness.