Closed-loop framework uses LLM analysis of policy rollouts to auto-correct structured policies, improving imitation learning by up to 15%
The work proposes a closed-loop framework that logs policy rollouts as semantically meaningful tabular data and prompts an LLM to generate diagnostic analysis code, iteratively identifying and correcting suboptimalities in structured policies without human instruction; on car racing and door opening tasks, it improves imitation learning performance by up to 15% over zero-shot LLM-generated structures and requires 75% less compute to achieve the same reinforcement learning performance.