# Audio-Visual Reasoning

The authors present OmniSeek, an agentic framework that makes evidence acquisition part of reasoning: the model dynamically decides whether to look or listen and over which temporal window within a multi-turn protocol, appends retrieved raw audio or visual segments back into context, cold-starts this behavior by supervising on the synthesized OmniTraj-170K corpus of multi-hop Chain-of-Thought trajectories, then optimizes the policy via two-stage reinforcement learning with verifiable rewards, and adds an Audio-Visual Necessity objective that rewards successful trajectories whose reasoning depends on both modalities, with experiments reporting adaptive cross-modal evidence seeking and consistent gains in audio-visual reasoning.

The authors present OmniSeek, an agentic framework that makes evidence acquisition part of reasoning: the model dynamically decides whether to look or listen and over which temporal window within a multi-turn protocol, appends retrieved raw audio or visual segments back into context, cold-starts this behavior by supervising on the synthesized OmniTraj-170K corpus of multi-hop Chain-of-Thought trajectories, then optimizes the policy via two-stage reinforcement learning with verifiable rewards, and adds an Audio-Visual Necessity objective that rewards successful trajectories whose reasoning depends on both modalities, with experiments reporting adaptive cross-modal evidence seeking and consistent gains in audio-visual reasoning.

Diagnosing multi-hop reasoning difficulties and partial decoupling in the local optimization of perception and reasoning objectives in omni-modal LLMs, this work proposes a text-centric post-training paradigm in which text-only training carries the main reasoning optimization and reduced-data native audio-visual RL then refines perception; the best text-only configuration (supervised fine-tuning followed by RL) raises Qwen2.5-Omni-7B's geometric mean of nine reasoning scores by 25.83% over the base model with 56.6% fewer GPU-hours than the complete native audio-visual route, training on data synthesized entirely by a text-only LLM raises this geometric mean by 21.01%, and refinement restores perception above the base level while retaining 93.5% of the reasoning gain.

Diagnosing multi-hop reasoning difficulties and partial decoupling in the local optimization of perception and reasoning objectives in omni-modal LLMs, this work proposes a text-centric post-training paradigm in which text-only training carries the main reasoning optimization and reduced-data native audio-visual RL then refines perception; the best text-only configuration (supervised fine-tuning followed by RL) raises Qwen2.5-Omni-7B's geometric mean of nine reasoning scores by 25.83% over the base model with 56.6% fewer GPU-hours than the complete native audio-visual route, training on data synthesized entirely by a text-only LLM raises this geometric mean by 21.01%, and refinement restores perception above the base level while retaining 93.5% of the reasoning gain.