Omnimodal models mishear and missee without retraining: an inference-time nudge back to the relevant modality lifts CMM overall accuracy from 70.5% to 85.9%
Omnimodal large language models answer wrongly when audio and video interfere with each other, yet the question-relevant unimodal branch often still holds the right answer; a training-free inference-time intervention steers the fused representation back toward that branch on conflicting samples, raising CMM overall accuracy from 70.5% to 85.9%.