Feiyu Gavin Zhu

The work introduces Selection-based Structured Reasoning (SSR), which reformulates the free-form reasoning that multimodal agents generate before each action as a selection over pre-specified, reusable natural-language reasoning candidates scored by their likelihood given the current context, without an auxiliary task head, using teacher-forced prefilling and a shared context KV cache for parallel scoring; on seven multimodal search benchmarks with 2B and 4B models, across multiple reinforcement learning objectives and supervised fine-tuning, SSR reaches an average success rate competitive with leading search agents of the same scale while reducing per-turn reasoning latency by over 90% and total per-question model inference latency by 28-54%.

The work introduces Selection-based Structured Reasoning (SSR), which reformulates the free-form reasoning that multimodal agents generate before each action as a selection over pre-specified, reusable natural-language reasoning candidates scored by their likelihood given the current context, without an auxiliary task head, using teacher-forced prefilling and a shared context KV cache for parallel scoring; on seven multimodal search benchmarks with 2B and 4B models, across multiple reinforcement learning objectives and supervised fine-tuning, SSR reaches an average success rate competitive with leading search agents of the same scale while reducing per-turn reasoning latency by over 90% and total per-question model inference latency by 28-54%.

The work proposes a closed-loop framework that logs policy rollouts as semantically meaningful tabular data and prompts an LLM to generate diagnostic analysis code, iteratively identifying and correcting suboptimalities in structured policies without human instruction; on car racing and door opening tasks, it improves imitation learning performance by up to 15% over zero-shot LLM-generated structures and requires 75% less compute to achieve the same reinforcement learning performance.

The work proposes a closed-loop framework that logs policy rollouts as semantically meaningful tabular data and prompts an LLM to generate diagnostic analysis code, iteratively identifying and correcting suboptimalities in structured policies without human instruction; on car racing and door opening tasks, it improves imitation learning performance by up to 15% over zero-shot LLM-generated structures and requires 75% less compute to achieve the same reinforcement learning performance.