# BudgetMix RL

Multi-turn agentic reinforcement learning is widely used to strengthen the long-horizon behavior of large language model agents, yet trajectory-level success rates hide what it does to individual decisions. Across four interactive tasks, DeepCoder-MT, BFCL, InterCode-SQL and FrozenLake, models trained with GRPO are evaluated along two axes, sampling breadth and interaction depth: with more sampled trajectories the RL model consistently beats its base model, but when the turns allowed per trajectory are tightened that edge shrinks, disappears, and on some tasks reverses. Matched-state comparisons show better next-step actions on only 46.8% of DeepCoder-MT states and 42.8% of FrozenLake states, with the gain concentrated in error recovery and later trajectory stages.

Multi-turn agentic reinforcement learning is widely used to strengthen the long-horizon behavior of large language model agents, yet trajectory-level success rates hide what it does to individual decisions. Across four interactive tasks, DeepCoder-MT, BFCL, InterCode-SQL and FrozenLake, models trained with GRPO are evaluated along two axes, sampling breadth and interaction depth: with more sampled trajectories the RL model consistently beats its base model, but when the turns allowed per trajectory are tightened that edge shrinks, disappears, and on some tasks reverses. Matched-state comparisons show better next-step actions on only 46.8% of DeepCoder-MT states and 42.8% of FrozenLake states, with the gain concentrated in error recovery and later trajectory stages.