A correct answer is not a correct derivation: GRPO gets a noise-aware group weight that keeps learning at high reward noise
GRPO, a common reinforcement-learning recipe for reasoning models, judges a rollout only by whether its final answer is right, so a correct answer can hide unreliable reasoning, and a single corrupted reward can strongly distort group-relative advantages when a response group is highly imbalanced. S-GRPO minimizes the expected squared error between observed and latent advantages under a reward-noise model to obtain a closed-form, noise-aware group weight. Across several mathematical reasoning benchmarks and model scales it stabilizes training and keeps making progress under controlled reward noise where standard GRPO shows little meaningful learning.