Mamba loses fine detail in RGB-D saliency detection: liquid time constants and wavelet fusion put LWNet first on five benchmarks
Synopsis
RGB-D salient object detection has traded the limited receptive field of CNNs against the quadratic cost of Transformers; Mamba models global context at linear cost but flattens 2D features into 1D sequences and loses local structure. LWNet adds a Vision Liquid Time-Constant module on a VMamba backbone, using input-dependent decay to restore detail weakened by scanning, and a cross-modal wavelet fusion that splits RGB and depth features into low- and high-frequency bands for separate alignment. Against 18 methods on STERE, NLPR, NJU2K, DUTLF and SIP, LWNet ranks first on all five datasets.
Interpretation
RGB-D saliency detection gains a route that does not rest on a CNN or Transformer backbone: Mamba supplies global context, and a dedicated module puts back the local structure lost during scanning. Backbones for this task were either CNNs with limited receptive fields or Transformers with prohibitive cost; Mamba offers global modeling at linear complexity, but flattening 2D feature maps into 1D sequences severs spatial correlation between adjacent pixels, and its smoothing effect weakens high-frequency cues such as edges. Compared with 18 methods on STERE, NLPR, NJU2K, DUTLF and SIP, LWNet ranks first on all five datasets; on STERE its MAE is 0.029 and Sα is 0.922.
Local detail is restored by a module that treats feature extraction as a continuous state evolution, letting each pixel set its own update speed from local complexity. The Vision Liquid Time-Constant module (Vi-LTC) extends liquid time-constant networks, originally limited to vector inputs, to 2D visual modeling by reformulating the ODE solver with convolutional operators and iterating with an explicit Euler solver. Ablation on NJU2K and DUTLF shows the single-stream VMamba baseline with element-wise addition fusion at Fβ 0.932 and MAE 0.034 on NJU2K, rising to 0.940 and 0.029 with Vi-LTC and to 0.945 and 0.023 with CMWF added.
Cross-modal fusion no longer happens in a single domain: RGB and depth features are split into low- and high-frequency components and aligned separately. The Cross-Modal Wavelet Fusion module (CMWF) has two branches: a wavelet branch decomposes features into low- and high-frequency subbands and reconstructs them by inverse transform to keep edges, while a state-space branch handles global semantics and per-modality enhancement. Replacing CMWF with the fusion strategies of HINet and EATNet on DUTLF, LWNet reaches Fβ 0.963 and MAE 0.019 versus 0.951 and 0.024 for HINet and 0.950 and 0.025 for EATNet, at 40.9M parameters and 64.5G FLOPs.
Perspective
The demonstrated scope is five natural-image RGB-D saliency benchmarks and two remote-sensing benchmarks; the remote-sensing datasets contain no depth maps, so the depth branch is removed and only the wavelet component is kept for evaluation. Inputs are resized to 352×352 and training uses a single GeForce RTX 4080.
Vi-LTC accuracy varies with solver precision: RK4 reaches Fβ 0.942 on NJU2K versus 0.938 for Euler, but FLOPs rise from 5.7G to 14.3G, while S-LTC reaches 0.945 at 7.3G. Failure cases concentrate on RGB objects with high internal contrast paired with poor depth maps, and on low RGB contrast against the background with equally poor depth.