Multimodal trackers stop correcting two streams in parallel: CCLT lets the second correction read the freshly updated state, training 0.212% of parameters
Synopsis
RGB-X tracking pairs visible frames with thermal, depth or event data, but paired annotations are far scarcer than RGB pretraining data, so only a small set of parameters is usually tuned. Earlier bidirectional cross-modal corrections computed both directions side by side from uncorrected input states, so the second correction could not use the first one's result. CCLT inserts low-rank exchange modules at blocks 6 and 10 of a frozen backbone, letting one direction correct first and the other read the freshly corrected state; against a parameter-matched parallel version, mean LasHeR SR rises by 1.980 points (95% interval [0.360, 2.690]), with 196.6K trainable parameters, 0.212% of the 92.7M-parameter tracker.
Interpretation
Multimodal trackers add thermal, depth or event imagery to visible frames to hold a target under illumination change or fast motion; because such paired data are scarce, the practice is to freeze a pretrained backbone and train only a few adaptation parameters. CCLT changes the order in which the two directional cross-modal corrections are computed: one direction updates its own state first, and the other then reads that updated state. Earlier parameter-efficient bidirectional designs placed two one-way corrections side by side at the same site, both starting from the states entering that site, so the reverse correction merely repeated a one-hop update and could not build on the forward one. Under a three-seed RGB-T protocol with identical parameter count, data split, optimizer, schedule and validation-only checkpoint selection, changing only what the second correction reads, from the incoming state to the corrected state, raises mean LasHeR SR by 1.980 points, with a 95% interval of [0.360, 2.690].
In implementation, the exchange modules sit at only two of the 12 ViT backbone blocks, blocks 6 and 10; at one site the thermal, depth or event stream corrects the RGB stream first, and at the other the RGB stream corrects the auxiliary stream first, so each modality initiates one relay. Each directional correction is a low-rank residual: the state is projected down to rank 8, passed through GELU and mapped back to 768 dimensions, so the added trainable volume is small; one set of low-rank key/value factors is shared between the RGB and auxiliary streams for within-stream adaptation. With all inherited parameters frozen, the trainable count is 196.6K, 0.212% of the 92.7M-parameter tracker, of which the exchange modules account for only 49,152; under the project-controlled network-forward-only timing, inference runs at 106.8 FPS.
Perspective
The demonstrated scope is RGB-T, RGB-D and RGB-E tracking on five public benchmarks: LasHeR and RGBT234, DepthTrack and VOT-RGBD2022, and VisEvent, with one checkpoint trained per modality; the primary controlled comparison is LasHeR mean SR in RGB-T.
Missing modalities remain a clear weakness: removing RGB or thermal lowers LasHeR SR by 38.09 and 19.27 points respectively; the authors list missing-modality augmentation, reliability-aware routing and sparse correction for partially observed or spatially misaligned inputs as future work. The intervals describe test-sequence sampling variation conditional on the two fixed checkpoints and do not estimate variation across training seeds.