OmniSeek lets an Omni-LLM decide when to look or listen across turns, consistently improving audio-visual reasoning on multimodal benchmarks
The authors present OmniSeek, an agentic framework that makes evidence acquisition part of reasoning: the model dynamically decides whether to look or listen and over which temporal window within a multi-turn protocol, appends retrieved raw audio or visual segments back into context, cold-starts this behavior by supervising on the synthesized OmniTraj-170K corpus of multi-hop Chain-of-Thought trajectories, then optimizes the policy via two-stage reinforcement learning with verifiable rewards, and adds an Audio-Visual Necessity objective that rewards successful trajectories whose reasoning depends on both modalities, with experiments reporting adaptive cross-modal evidence seeking and consistent gains in audio-visual reasoning.