Raw radar spectra skip the point-cloud step: L-SPMAE pretraining lets a generic detector beat radar-specific detectors on K-Radar
Synopsis
Radar 3D perception has long compressed raw spectra into sparse point clouds or compact Doppler statistics before supervised training. L-SPMAE instead uses masked self-supervision: whatever remains reconstructable under localized corruption counts as signal, and spectral leakage supplies that corruption, so pretraining runs directly on unreduced radar spectra. On K-Radar, L-SPMAE with the RAD representation reaches 55.9/50.1 mAP after fine-tuning, above the best point-cloud self-supervised baseline in the table at 50.8/45.0.
Interpretation
The default radar pipeline first reduces spectra by hand into sparse point clouds or compact Doppler descriptors and then trains a detector on them; L-SPMAE skips that step and pretrains self-supervised directly on unreduced radar spectra. Under fully supervised training, richer spectral representations do worse: on the official K-Radar split, RAE and RAE-d fall 1.6/1.5 and 3.2/2.8 mAP below point clouds. After pretraining and fine-tuning the ordering reverses: L-SPMAE on RAE reaches 53.7/48.4 mAP, 2.9/3.4 above the best point-cloud self-supervised baseline in the table, and L-SPMAE on RAD reaches 55.9/50.1 mAP, 5.1/4.7 above all point-cloud baselines.
The method defines signal as the spectral component still reconstructable under localized corruption and noise as the component that is not, so it needs no prior on what signal looks like. Random-mask SPMAE already beats the point-cloud self-supervised comparisons, and adding leakage corruption in L-SPMAE raises RAE by a further 0.6/0.3 mAP. In the three-class detection setting on the official K-Radar split, with the PTv3 backbone and RAD representation fixed, SPMAE improves on supervised training by 5.4/4.8 and L-SPMAE adds another 1.6/2.2 over SPMAE.
Doppler cannot simply be stacked as channels; it must be an explicit spatial axis so the encoder can model dependencies along the Doppler direction. RAE-d, which stacks Doppler bins as channels, underperforms RAE with mean Doppler power after fine-tuning, so stacking brings no benefit. With the x-y-D RAD representation, L-SPMAE reaches 55.9/50.1 mAP on the official K-Radar split, restoring the expected ordering in which richer representations do better after pretraining.
Perspective
The demonstrated scope is 3D object detection on K-Radar and RADIal plus occupancy prediction and multi-object tracking on K-Radar; all results come from the authors' own pretraining and fine-tuning pipeline, with the generic 3D detector PTv3 as backbone and a sparse encoder used to confirm the same trend.
The authors note that processing dense radar spectra costs substantially more than point-cloud methods, with slower training and inference; they plan token pruning to cut overhead and to explore self-supervised methods beyond MAE. Gains on RADIal are smaller, which the authors attribute to that dataset's lower scene diversity and complexity.