# Audio-Visual Large Language Models

2 related articles

Subscribe to updates

arXiv Oct 5, 2026
RED, a training-free decoding method, raises accuracy by up to 7% on three audio-visual hallucination benchmarks at about 1.5x standard time to first token
The work introduces Relevant Evidence Decoding (RED), a training-free method that uses pointwise mutual information to decompose the joint audio-visual prediction into audio, video, and residual interaction components, then runs a question-only inference pass to determine the required evidence type and augments the original prediction with the corresponding contribution; across three audio-visual hallucination benchmarks and three AV-LLMs, RED improves over standard decoding by up to 7% on CMM, 6.3% on AVHBench, and 3.8% on SVHalluc, with an average relative time to first token of 1.5x standard decoding.
The work introduces Relevant Evidence Decoding (RED), a training-free method that uses pointwise mutual information to decompose the joint audio-visual prediction into audio, video, and residual interaction components, then runs a question-only inference pass to determine the required evidence type and augments the original prediction with the corresponding contribution; across three audio-visual hallucination benchmarks and three AV-LLMs, RED improves over standard decoding by up to 7% on CMM, 6.3% on AVHBench, and 3.8% on SVHalluc, with an average relative time to first token of 1.5x standard decoding.
Daegu Gyeongbuk Institute of Science and Technology Sungkyunkwan University
Hyunjae Ra（First author）
Relevant Evidence Decoding Audio-Visual Large Language Models Hallucination Mitigation
arXiv Oct 2, 2026
SECRET intercepts cross-modal interference at the question relay, lifting audio-visual LLM accuracy by up to 18.0 points on CMM
Through path-intervention and representation analyses, this work identifies a question-relay mechanism in audio-visual large language models, where question-position hidden states carry both required-modality evidence and interfering-modality cues, and proposes SECRET, a training-free method that steers question states toward required-source evidence using source-conditioned positive and negative question representations, achieving higher accuracy than prior training-free methods on CMM and AVHBench across three AVLLMs, with gains of up to 18.0 and 7.1 percentage points over base models, and reducing distractor overlap in modality-specific captioning.
Through path-intervention and representation analyses, this work identifies a question-relay mechanism in audio-visual large language models, where question-position hidden states carry both required-modality evidence and interfering-modality cues, and proposes SECRET, a training-free method that steers question states toward required-source evidence using source-conditioned positive and negative question representations, achieving higher accuracy than prior training-free methods on CMM and AVHBench across three AVLLMs, with gains of up to 18.0 and 7.1 percentage points over base models, and reducing distractor overlap in modality-specific captioning.
Fudan University Harbin Institute of Technology
Yu Zhang（First author）
Secret (SourcE-Conditioned RElay sTeering)Source-Confused Grounding Hallucination Audio-Visual Large Language Models AVHBench

You’re all caught up

RED, a training-free decoding method, raises accuracy by up to 7% on three audio-visual hallucination benchmarks at about 1.5x standard time to first token

SECRET intercepts cross-modal interference at the question relay, lifting audio-visual LLM accuracy by up to 18.0 points on CMM