Hyperspectral imaging stops trading highlights for shadows: fusing exposures in the encoded measurement domain lifts spectral dynamic range by 16.0 dB
Synopsis
Computational hyperspectral imaging must hold highlights and shadows in one shot, but the encoding mask's channels have unequal spectral responsivities, so no single exposure keeps every channel in its reliable range. A Zhejiang University team moves multi-exposure fusion ahead of spectral inversion, into the encoded measurement domain, where an analytical reliability prior and a learned channel-specific adjustment jointly set fusion weights, and measurement-domain spectral-angle supervision constrains relative channel responses. On a 9-channel broadband-encoded camera, an optical-density target gives a spectral reconstruction dynamic range of 42.8 dB, 16.0 dB above the best fixed single exposure at 26.8 dB.
Interpretation
Multi-exposure fusion now happens before spectral inversion, producing one high-dynamic-range measurement in the encoded measurement domain that is then passed to the spectral reconstruction network. Earlier multi-exposure HDR methods mainly target the final display image, and RAW-domain fusion also aims at a visual image rather than an intermediate measurement for downstream algorithmic inversion. On HDR test data synthesized from the ICVL dataset, across 20 test scenes and four three-exposure configurations, measurement-domain RMSE and SAM drop by about 49% relative to the best competing learning-based method.
Each exposure is first scored by a shared reliability estimator, and an analytical prior plus a learned channel-specific adjustment together produce fusion weights that vary across both space and encoding channel. An analytical hat-shaped prior only favors the intermediate response range by signal level and cannot capture the differing hardware deviations of individual encoding channels. Ablation shows channel-specific weighting raises spectral PSNR from 44.180 dB with channel-shared weighting to 44.606 dB, and combining the prior with the learned adjustment beats either component alone.
On the 9-channel broadband-encoded camera, full-field sensor-response characterization and a channel-aware reliability gate suppress measurements beyond the reliable range. An individual CMOS encoding channel can depart from linearity before nominal saturation, and lookup-table correction leaves substantial residual deviation at high signal levels. On a 24-patch ColorChecker, two and three exposures reach 38.341 dB and 38.992 dB, above the best single exposure at 35.792 dB, with a spectral angle of 0.0285 rad at patch P1.
Perspective
The demonstrated scope is static, spatially aligned scenes on a 9-channel broadband-encoded snapshot hyperspectral camera producing 31 bands over 400–700 nm; the quantitative dynamic-range evaluation uses the 20 peripheral patches of an OECF target spanning optical density 0.02–4.19, and the darkest patch was excluded from quantitative analysis because its signal was too weak.
The current method does not address inter-exposure motion, and the authors list motion compensation and co-design with adaptive optical modulation hardware as future work; at low transmittance both fusion methods perform similarly and eventually fall below the longest exposure, suggesting low signal-to-noise ratios may limit reliable fusion; because the brightest patch still met the fidelity criterion, the measured spectral reconstruction dynamic range is bounded by the tested optical-density range.