Visible and infrared stop being fused in one shot: staged modality control reaches 88.6% mAP50 on M3FD

Visible-infrared detection is organized as a stage-constrained modality-control chain rather than a side-by-side stack of frequency enhancement, cross-modal fusion and multi-scale aggregation. Earlier fusion detectors mainly handle dual-modal information in the fusion encoder or decoder, with limited attention to whether modality structure is compressed before fusion or infrared context is diluted after it. On the same M3FD 80%/20% split, the full model reaches 88.6% mAP50 and 63.7% mAP50:95, against 82.6% and 56.5% for the same-backbone RGB+IR RT-DETR baseline; three runs with different seeds give 88.5±0.2 and 63.6±0.2 versus 82.5±0.2 and 56.4±0.2.

A lightweight frequency adapter before fusion preserves spatial texture and frequency structure for each modality at the shared backbone's multi-scale outputs. Shared-backbone or compact two-stream designs emphasize parameter efficiency, so weak-texture objects, small-scale boundaries and weak thermal regions may be compressed before fusion. In the M3FD ablation, adding this adapter alone raises mAP50 from 82.6% to 85.0% and mAP50:95 to 59.0%; removing the frequency branch drops it to 87.0%, while the default averaged Haar design reaches 88.6% at 48.8M parameters.

A cross-modal interaction module during fusion redistributes dual-modal responses through global modality gating, local complementary compensation and channel-sequence correction. Global weighting, local attention or query-level interaction struggle to constrain each modality's useful contribution across regions, channels and scales, so strong background texture and slight misalignment can be amplified. On M3FD, adding global gating, local complementarity and channel correction in turn lifts mAP50 from 85.0% to 85.7%, 86.3% and 86.9%, and the full interaction module is 1.9 points above the variant without it.

A contextual kernel-guided fusion after fusion injects infrared features as conditional context during top-down and bottom-up multi-scale propagation, driven by the current scale state. Conventional neck or encoder fusion treats infrared information as part of the fused input, with less attention to dynamically injecting infrared context according to scale-specific feature states. On M3FD, the infrared compensation branch alone raises mAP50 from 86.9% to 88.6%, 1.7 points above the variant without it; using both RGB and infrared compensation branches brings no further gain and raises parameters to 50.3M.

The demonstrated scope is the full model on four public RGB-IR datasets: M3FD, DroneVehicle, VEDAI and FLIR reach 88.6%, 84.3%, 86.7% and 81.5% mAP50, with M3FD using the public 3360/840 train/test split. Detailed module ablations concentrate on M3FD, while the other three datasets mainly support full-model evaluation and cross-dataset analysis. Efficiency figures come from FP32 inference on an RTX 3090 with batch size 1 and 640×640 inputs, where the full model runs at 34 FPS with 48.8M parameters, and should not be read as embedded real-time performance. Cross-dataset experiments use a source-only zero-shot setting with a fixed detection head and a predefined shared-class projection.

The gates are optimized only through the detection objective, and the perturbation experiments measure detection-performance changes without verifying whether gate values track independently measured modality quality, so reliability-inspired gating describes an intended control role rather than calibrated probabilities. The robustness analysis fixes one trained model and applies selected test-time perturbations on the M3FD test set, does not compare all detectors under identical corruptions, and zero-filled dropout and zero-padding shifts do not fully reproduce real sensor failures or registration errors. The cross-dataset table does not include source-trained checkpoints for other detectors, so it is not a transferability ranking. M3FD has no separate official validation set, so 88.6% should be read as a public-protocol development result.