# HuatuoGPT-3

The work proposes One-stage Policy Optimization (OnePO), which treats teacher outputs as transient guidance rather than persistent training targets: a probability floor and gradient rescaling strengthen learning on low-probability teacher tokens, and Teacher Retirement discards a teacher output once the current policy's reward surpasses it. With only 20K medical samples, OnePO reaches 67.2 on HealthBench (Total), outperforming SFT+RL and pure RL under the same teacher, and scales to the open-source HuatuoGPT-3 series, whose 27B variant reaches 71.4 on HealthBench Professional.

The work proposes One-stage Policy Optimization (OnePO), which treats teacher outputs as transient guidance rather than persistent training targets: a probability floor and gradient rescaling strengthen learning on low-probability teacher tokens, and Teacher Retirement discards a teacher output once the current policy's reward surpasses it. With only 20K medical samples, OnePO reaches 67.2 on HealthBench (Total), outperforming SFT+RL and pure RL under the same teacher, and scales to the open-source HuatuoGPT-3 series, whose 27B variant reaches 71.4 on HealthBench Professional.