Reward the search, not just the answer: process supervision lifts retrieval recall and holds up out of domain
Search agents for multi-hop information seeking are trained on step-level judgments of each search action instead of only on final-answer correctness. RAG-Gym casts search as a Markov decision process and compares architecture, parameter optimization and action evaluation, combining a reasoning-reflection architecture, preference optimization and critic-guided inference into Re2Search++. Trained on HotpotQA, it beats outcome-supervised baselines on the unseen Bamboogle benchmark with higher retrieval recall at comparable or fewer search steps, and a critic trained on Llama-3.1-8B raises GPT-4o-mini MedQA accuracy.