Remote sensing super-resolution stops feeding dense low-res maps: object slots lift PSNR by 2.03 dB over DiT4SR on UCMerced
Synopsis
Remote sensing super-resolution previously injected low-resolution observations as dense spatial conditions into a diffusion Transformer, so small foreground objects were easily diluted by dominant backgrounds. MOSAIC organizes degraded observations into compact object-level and layout-level structural slots, fuses them across resolutions, and retrieves recurring structural priors from a shared prototype memory, then injects them into a frozen DiT4SR backbone through lightweight residual modulation. On UCMerced, AID and RSSCN7, PSNR is best at both scales, exceeding DiT4SR by 2.03 dB on UCMerced, 4.54 dB on AID and 1.49 dB on RSSCN7, with the lowest LPIPS on RSSCN7 and AID.
Interpretation
Conditioning for remote sensing super-resolution moves from dense low-resolution features to compact object-level structural slots, so foreground objects and structural boundaries are no longer diluted by background. Earlier DiT-based methods injected low-resolution observations as dense spatial conditions, entangling foreground objects with dominant backgrounds and weakening small targets and structural boundaries during denoising. Across UCMerced, AID and RSSCN7 at two scales, PSNR is best; relative to DiT4SR the gains are 2.03 dB on UCMerced, 4.54 dB on AID and 1.49 dB on RSSCN7.
Slots are extracted at the original and resized scales, where coarse slots keep stable layouts and fine slots keep textures, boundaries and small objects, and the two are fused in slot space. Single-scale dense conditions offer structural evidence at only one resolution, and fixed averaging of multiple scales improves little over the single-scale baseline. On the UCMerced validation set, full multi-scale fusion reaches 24.2510 dB PSNR, 0.6386 SSIM and 0.2210 LPIPS, above the 24.2440 dB of inter-scale fusion alone.
Fused slots retrieve image-guided prototype context to supplement image-specific evidence erased by severe degradation, and the retrieved context is added residually rather than replacing the slots. Image-specific slots do not explicitly summarize structures shared across samples, while direct quantization or feature replacement can discard image-specific detail. The prototype memory is built offline from training slots by KMeans and stays frozen, with only the prototype cross-attention projections and SlotFiLM adapter updated; in ablation this module adds 0.0079 dB PSNR, 0.0002 SSIM and a 0.0033 LPIPS reduction under that protocol.
Enhanced slots are pooled and normalized into a global condition, and SlotFiLM predicts layer-wise channel scale and shift terms that residually modulate frozen DiT4SR blocks. There was no explicit path for injecting compact structural conditions into a frozen diffusion Transformer, and dense single-scale conditions retained excessive background information. Placement ablation shows the best PSNR of 24.2439 dB when SlotFiLM is injected into early and middle DiT blocks; zero-initialized projections make the adapter initially close to identity, and residual modulation preserves DiT token length and generation path.
Perspective
Results cover UCMerced, AID and RSSCN7 at two scale settings with full-reference and no-reference metrics, where MANIQA and CLIP-IQA are evaluated at only one scale. The method targets structure-sensitive image restoration, adapts a frozen DiT4SR backbone, and builds the prototype memory offline from training slots that stay frozen and are used at both adaptation and inference. The authors propose developing dynamic and more transferable structural priors as future work.
The prototype memory is built offline from the training distribution, which the authors note may limit adaptation to unseen sensors and degradations. No-reference MANIQA and CLIP-IQA results vary across datasets while remaining competitive. In the slot-number ablation, PSNR varies non-monotonically and peaks at the chosen slot count before decreasing.