Jiaying Zhou

GRPO scores every rollout of a prompt against one shared group-mean baseline, yet rollouts of the same prompt often follow different solution strategies with different reward distributions. This work defines the subgroup offset as a subgroup mean's deviation from the global mean and shifts the baseline along it by a single signed coefficient: the positive sign pulls credit toward a rollout's standing inside its subgroup, the negative sign pulls it toward the subgroup's overall quality. On Qwen3-8B-Base and Olmo3-7B-Base across four competition-level math benchmarks, both directions give modest but consistent gains over an anchor that shares the same strategy-conditioned sampling and differs only in the baseline, concentrated in pass@16, with the negative direction stronger.

GRPO scores every rollout of a prompt against one shared group-mean baseline, yet rollouts of the same prompt often follow different solution strategies with different reward distributions. This work defines the subgroup offset as a subgroup mean's deviation from the global mean and shifts the baseline along it by a single signed coefficient: the positive sign pulls credit toward a rollout's standing inside its subgroup, the negative sign pulls it toward the subgroup's overall quality. On Qwen3-8B-Base and Olmo3-7B-Base across four competition-level math benchmarks, both directions give modest but consistent gains over an anchor that shares the same strategy-conditioned sampling and differs only in the baseline, concentrated in pass@16, with the negative direction stronger.