# Hindsight Credit Assignment

Decision models are usually judged by how well they act, yet the same models are pulled toward an easier subgoal when acting on ALFWorld. Used instead as a teacher that sees the whole trajectory after an episode and is asked whether each action helped, JEV raises a Qwen2.5-1.5B policy trained with GRPO from 72.8% to 82.9% success on ALFWorld and from 56.8% to 71.4% on WebShop, while JEV as an actor completes only 42.9% of ALFWorld tasks.

Decision models are usually judged by how well they act, yet the same models are pulled toward an easier subgoal when acting on ALFWorld. Used instead as a teacher that sees the whole trajectory after an episode and is asked whether each action helped, JEV raises a Qwen2.5-1.5B policy trained with GRPO from 72.8% to 82.9% success on ALFWorld and from 56.8% to 71.4% on WebShop, while JEV as an actor completes only 42.9% of ALFWorld tasks.

Multi-turn LLM agent training now ties exploration and credit assignment to one hindsight score: turns that lower the reference-answer likelihood are revisited and re-branched from the same parent, and inverse weights fold the outcomes back into the original policy, giving the highest aggregate exact match across seven QA benchmarks and three Qwen backbones.

Multi-turn LLM agent training now ties exploration and credit assignment to one hindsight score: turns that lower the reference-answer likelihood are revisited and re-branched from the same parent, and inverse weights fold the outcomes back into the original policy, giving the highest aggregate exact match across seven QA benchmarks and three Qwen backbones.