FastRL brings about 2.07x training speedup and roughly 1.64% average accuracy gain to GRPO-style methods via advantage-aware pruning and adaptive sampling
The work proposes FastRL, a plug-and-play reinforcement learning framework that selectively preserves high-advantage trajectories while maximizing inter-trajectory gradient diversity through an advantage-aware pruning strategy, and dynamically adjusts sampling scale across training stages based on historical pruning distributions via an adaptive rollout sampling mechanism, achieving an average 2.07x training speedup on Geometry3K and GeoQA8K-R1V along with approximately 1.64% improvement in average accuracy on visual reasoning benchmarks, and integrating seamlessly into GRPO, DAPO, and GSPO variants.