Factory calibration of multi-camera rigs drifts with heat and vibration: RECALL re-estimates a 10-camera rig's parameters in one 0.12 ms pass
Synopsis
Movable multi-camera rigs ship pre-calibrated, but repositioning, thermal cycling and vibration make that calibration go stale, and classical bundle adjustment needs a fresh multi-view capture plus a global optimization whose cost grows with the rig, ruling out real-time use. RECALL anchors on the factory calibration and, from a brief observation of a known object, predicts in a single feed-forward pass a shared global rotation of the rig plus small per-camera drift, solving the shared rotational gauge in-network with a learned Gauss-Newton step. On a synthetic 10-camera benchmark its parameter errors are lower than per-camera and rig-coupled joint bundle adjustment, at 0.12 ms per rig.
Interpretation
A multi-camera rig that has been repositioned no longer needs an offline multi-view capture and global optimization; one feed-forward pass yields each camera's current intrinsics and extrinsics. Factory calibration drifts through thermal expansion and contraction, mechanical shock and vibration from transport or re-mounting, mount wear, and lens smudges, and both per-camera and rig-coupled joint bundle adjustment require a fresh capture and an optimization whose cost grows with the number of cameras. On the synthetic 10-camera benchmark RECALL reaches 0.26 degrees absolute rotation error and 5.43 px focal error, against 0.47 degrees and 8.88 px for rig-coupled joint bundle adjustment and 2.30 degrees and 503.88 px for per-camera bundle adjustment; RECALL takes 0.12 ms per rig.
The prediction is split into a shared global rotation of the whole rig and a small per-camera drift from the factory values, keeping the drift part small and regressable. The shared global rotation is the ill-conditioned quantity coupling all cameras, so a pooled regressor for it leaves an absolute-orientation residual even when the inter-camera geometry is accurate. The network takes the factory calibration and 2D-to-3D correspondences of a known object, recurrently reprojects its current estimate and applies cross-camera attention, and at each step runs one Gauss-Newton iteration with learned confidence and damping restricted to the shared rotation increment; training data are synthesized on the fly by sampling a global rotation and per-camera drift and projecting the known object, so no captured calibration dataset is needed.
Perspective
The demonstrated scope is a movable rig that ships pre-calibrated, whose cameras are rigidly mounted and whose global reorientation is bounded, with input being a brief observation of a known calibration object and its 2D-to-3D correspondences; fiducial detection is upstream. On real data the drift and global reorientation are injected at controlled magnitudes onto real captured structure.
A model is trained per rig and object configuration; very large reorientations such as free axial rotation or near-90-degree tilt fall outside the bounded-rotation assumption; gross correspondence outliers degrade accuracy without a robust front-end; and focal length is only weakly observable from a single snapshot, where the loss defers to the factory prior, with multiple snapshots needed to sharpen it.