# GRPO

Online Rubric Self-Distillation (ORSD) has the current policy generate natural-language process rubrics from differences among a group of its own trajectories, lets a frozen verifier pick the bundle whose rubric satisfaction best correlates with task success, and uses that bundle as privileged context: the log-likelihood gap for the same tokens with and without the rubric becomes a token-level correction to the trajectory-level GRPO advantage. On ALFWorld and WebShop, Qwen3-1.7B and Qwen3-4B reach the best overall results among the compared methods under the same environment-interaction budget.

Online Rubric Self-Distillation (ORSD) has the current policy generate natural-language process rubrics from differences among a group of its own trajectories, lets a frozen verifier pick the bundle whose rubric satisfaction best correlates with task success, and uses that bundle as privileged context: the log-likelihood gap for the same tokens with and without the rubric becomes a token-level correction to the trajectory-level GRPO advantage. On ALFWorld and WebShop, Qwen3-1.7B and Qwen3-4B reach the best overall results among the compared methods under the same environment-interaction budget.

Traffic signal control has relied on structured traffic states from dedicated perception pipelines, or on converting simulator images into intermediate representations before deciding. Traffic-VL is an eight-billion-parameter vision-language model that maps synchronized multi-view images and natural-language phase descriptions directly to phase choices, trained with SynTra-V, which turns physics-based simulation into semantically audited photorealistic surveillance imagery. It matches the structured-state Traffic-R1 on five closed-loop benchmarks, reaches 82.1% phase-selection accuracy on real surveillance images, and over three weeks at ten intersections reduced average queue length by 10.4% relative to historical manual operation.

Traffic signal control has relied on structured traffic states from dedicated perception pipelines, or on converting simulator images into intermediate representations before deciding. Traffic-VL is an eight-billion-parameter vision-language model that maps synchronized multi-view images and natural-language phase descriptions directly to phase choices, trained with SynTra-V, which turns physics-based simulation into semantically audited photorealistic surveillance imagery. It matches the structured-state Traffic-R1 on five closed-loop benchmarks, reaches 82.1% phase-selection accuracy on real surveillance images, and over three weeks at ten intersections reduced average queue length by 10.4% relative to historical manual operation.

Long-horizon language-model agents accumulate ever-longer interaction histories, so context compaction is necessary, yet existing learned compaction methods often depend on downstream task outcomes or solver behavior. SumOne is a standalone 2-billion-parameter summarizer trained with a task-agnostic reward built from masked reconstruction and optimized with GRPO, using only raw trajectories and no reference summaries or downstream task execution. On DeepSearchQA it cuts cumulative agent input from 2.83M to 591K tokens with F1 0.48 versus 0.52 for full context; on SWE-bench Verified it reaches a 0.59 resolved rate versus 0.63, with input falling from 4.10M to 1.09M tokens.

Long-horizon language-model agents accumulate ever-longer interaction histories, so context compaction is necessary, yet existing learned compaction methods often depend on downstream task outcomes or solver behavior. SumOne is a standalone 2-billion-parameter summarizer trained with a task-agnostic reward built from masked reconstruction and optimized with GRPO, using only raw trajectories and no reference summaries or downstream task execution. On DeepSearchQA it cuts cumulative agent input from 2.83M to 591K tokens with F1 0.48 versus 0.52 for full context; on SWE-bench Verified it reaches a 0.59 resolved rate versus 0.63, with input falling from 4.10M to 1.09M tokens.

In multi-turn tool-use reinforcement learning, reasoning accuracy can rise early and then fall below the non-agentic baseline; TACA traces this to negative aggregate credit on tool-call tokens and instead supervises the step where reasoning turns into a tool call. Against MT-GRPO it raises mean accuracy over six reasoning benchmarks by 4.61 and 3.61 percentage points for Qwen2.5-1.5B and 3B, while tool execution success during training stays at 86.28%.

In multi-turn tool-use reinforcement learning, reasoning accuracy can rise early and then fall below the non-agentic baseline; TACA traces this to negative aggregate credit on tool-call tokens and instead supervises the step where reasoning turns into a tool call. Against MT-GRPO it raises mean accuracy over six reasoning benchmarks by 4.61 and 3.61 percentage points for Qwen2.5-1.5B and 3B, while tool execution success during training stays at 86.28%.

Automated chest X-ray reports often read fluently while leaving unclear which image evidence supports which finding. ARIVE interleaves anatomical inspection with diagnostic reasoning before composing the report, trained on 159,416 multimodal chain-of-thought trajectories built from MIMIC-CXR. On MIMIC-CXR and the abnormality-only MIMIC-ABN it scored highest overall on clinical-efficacy measures among the compared models, and in blinded grading of 100 cases its report quality was close to Qwen3.5 (mean 3.47 versus 3.51, against 3.88 for reference reports).

Automated chest X-ray reports often read fluently while leaving unclear which image evidence supports which finding. ARIVE interleaves anatomical inspection with diagnostic reasoning before composing the report, trained on 159,416 multimodal chain-of-thought trajectories built from MIMIC-CXR. On MIMIC-CXR and the abnormality-only MIMIC-ABN it scored highest overall on clinical-efficacy measures among the compared models, and in blinded grading of 100 cases its report quality was close to Qwen3.5 (mean 3.47 versus 3.51, against 3.88 for reference reports).

Multimodal large language models can read text in an image accurately yet still fail to reason over the same problem as well as they do over plain text, a shortfall the authors call a modality reasoning-access gap. TIGER renders text-only math problems as images, mines cases the model solves from text but fails from the image, and post-trains on those image inputs with GRPO and related objectives. Trained only on synthetic rendered-text images, Qwen3-VL-8B rises from 54.5 to 65.8 on MathVerse, while Qwen2.5-VL-7B rises from 43.9 to 47.5 on MathVerse and from 22.2 to 25.7 on EMMA.

Multimodal large language models can read text in an image accurately yet still fail to reason over the same problem as well as they do over plain text, a shortfall the authors call a modality reasoning-access gap. TIGER renders text-only math problems as images, mines cases the model solves from text but fails from the image, and post-trains on those image inputs with GRPO and related objectives. Trained only on synthetic rendered-text images, Qwen3-VL-8B rises from 54.5 to 65.8 on MathVerse, while Qwen2.5-VL-7B rises from 43.9 to 47.5 on MathVerse and from 22.2 to 25.7 on EMMA.