Full-waveform LiDAR transformers reach real time: FWL-ToPM drops low-intensity background and gains nearly 9 F1 points at 53.2 FPS
Synopsis
Full-waveform LiDAR (FWL) records a complete return histogram for every pixel, so its data volume far exceeds a point cloud, and Transformer-based FWL models have run at only a few frames per second, short of real time. FWL-ToPM prunes low-intensity background patches by signal intensity and then merges the remaining patches along a curve that preserves spatial proximity, raising mean F1 on the Ghost-FWL benchmark from 0.503 to 0.592 while reaching 53.2 FPS. The companion FWLAug expands training samples while preserving each ray's time-of-flight waveform and the frustum geometry, adding a 4.6-point F1 gain on an out-of-distribution set of unseen scenes.
Interpretation
A full-waveform LiDAR transformer improves recognition accuracy while holding real-time throughput, rather than trading accuracy for speed. Earlier FWL transformers were limited by the large data volume of a full per-pixel histogram and ran well below real-time rates, with FWL-MAE on Ghost-FWL reaching only 6.54 frames per second. On the Ghost-FWL benchmark, FWL-ToPM reaches a mean F1 of 0.592 at 53.2 FPS, about 9 points above the prior best of 0.503 and more than 8 times faster.
The speedup comes from selecting tokens by physical structure: patches are ranked by their maximum signal intensity, the lowest-intensity 3D patches are discarded, and the rest are merged. Token pruning methods built for images and video assume information is spread evenly over 2D space and would discard the sparse high-intensity returns that carry object information in FWL data. The ablation gives a mean F1 of 0.555 with neither component, 0.591 with pruning only, 0.565 with merging only, and 0.592 with both; before merging, the sparse patches are serialized in Morton order, and swapping in a Hilbert curve gives a mean F1 of 0.589.
Perspective
Evaluation centers on voxel-wise classification on the Ghost-FWL dataset, with input cropped to one quarter of the full frame and a sliding window used to reconstruct full-frame predictions at inference; generalization is tested only on 500 frames of unseen scenes captured with the same FWL sensor platform, acquisition configuration and annotation protocol. The authors list validation across sensors, resolutions and weather conditions as future work.
Intensity-based pruning may wrongly remove weak signals from distant or low-reflectance objects or from adverse weather, and pruned regions are labeled as noise at inference, so such removal can directly cause false negatives; FWLAug is a regularization method that preserves waveform structure rather than an optical simulator, and it does not model distance-dependent intensity attenuation or recompute optical interactions between pasted glass and shifted objects; efficiency is measured only as model-level throughput, not sensor-to-decision latency or embedded deployment.