User simulators now write a private review each turn: assistant RL keeps its score and halves the compute
A user simulator now writes a first-person evaluation of the assistant before producing its next line, including a 1-to-5 rating and a critique-or-praise judgment; on real user-ChatGPT logs it tracks human users better than prompted simulators of the same size, and it lets multi-turn RL train assistants to comparable quality with less than half the compute.