Skip to main content
Back to timeline
Applied Physics BSource publication:

Turbulence video restoration gains an inspectable optical constraint: GridPSF forward fit reaches 29.49 dB, but restoration gains do not hold up

Synopsis

GridPSF adds an inspectable forward constraint to the MambaTM restoration network's training graph: from the observed clip and the host's latent field it predicts an 8×8 grid of 35 piston-free Zernike coefficients per cell, then blends unit-mass point-spread functions rendered by a Fraunhofer model to fit the turbulent observation. On held-out ATSyn-dynamic videos, forward consistency reaches 29.49 dB PSNR and 0.8882 SSIM, 3.95 dB above the 25.54 dB identity floor of using the clean target as the prediction. One validation-selected matched-host checkpoint pair gives +0.419 dB PSNR and +0.0022 SSIM, but LPIPS rises by 0.0017, and a five-seed control detects no average restoration gain.

Source-provided article image: GridPSF: an inspectable Zernike-parameterized forward regularizer for synthetic turbulence video restoration

Interpretation

A turbulence video restoration network now carries an explicit optical forward constraint during training, and its fit can be inspected cell by cell instead of inferring the image-formation operator from restoration scores alone. MambaTM previously guided restoration with a four-channel latent phase-distortion representation whose coordinates do not directly provide pupil phase, point-spread-function width, anisotropy or modal energy. On held-out ATSyn-dynamic test videos, applying the inferred field to the clean target gives 29.49 dB PSNR and 0.8882 SSIM against the turbulent observation; on the same 100 videos, using the clean target itself as the prediction gives only 25.54 dB, so the fitted operator sits 3.95 dB above that floor.

During training the constraint concatenates the observed clip with the host's latent phase distortion, predicts a time-varying Zernike coefficient grid, and renders local point-spread functions whose blur is blended across the image. Rendering uses a scalar Fraunhofer model with a hard circular pupil, each cell's point-spread function is normalized to unit mass, and 64 cells are nodes of a four-neighbour graph whose message passing produces non-negative weights renormalized at every pixel into a partition of unity. The main model uses an 8×8 grid and 35 piston-free Zernike modes covering radial orders 1 through 7, with 16-frame clips, trained on 400 ATSyn-dynamic training clips, checkpoint-selected on 100 validation clips and evaluated on a separate 100-video test set.

A single matched-host checkpoint pair shows a positive restoration difference on the test set, but that gain is checkpoint-sensitive and does not by itself establish that the optical parameterization improves restoration. Only 22 of 54 matched validation checkpoints have a positive PSNR difference, the unweighted mean difference is −0.018 dB, and the 29000-step pair is the maximum. That pair raises PSNR by 0.419 dB and SSIM by 0.0022 across 100 held-out videos with positive differences in all three severity regimes, while LPIPS rises by 0.0017; across five seeds, comparisons with a same-information CNN and the unmodified host have seed-level intervals spanning zero.

Perspective

Quantitative restoration evaluation is carried out on the single synthetic ATSyn-dynamic benchmark, with MambaTM as the host that exposes the four-channel latent phase-distortion interface, and no external restoration method was retrained; forward consistency is evaluated on the central 192×192 crop of each frame, and the point-spread-function geometry diagnostic uses only the first four held-out clips in each severity regime. Measured turbulence appears only as a qualitative check on static OTIS printed charts, with no quantitative real-video gain and no moving natural-scene test.

Forward fit does not identify the generating wavefront, and the fitted coefficients are not calibrated atmospheric-wavefront estimates; in replayed synthetic turbulence each clip's own field beats wrong-clip and shuffled-cell controls on displacement, but displacement magnitude is underestimated and blur-width ranking and shape agreement remain poor. The main model also has a half-grid point-spread-function assignment shift and cross-cell temporal mixing, which limits interpretation of the per-cell maps; both the 29000-step and five-seed results use that implementation.