# Hierarchical Reinforcement Learning

Humanoid soccer robots have either covered only a few skills or skipped motion priors entirely and moved unnaturally. SoccerBot distills large-scale human soccer motion-capture data into a structured latent motion prior, letting a single skill-conditioned policy learn moving, dribbling, trapping and kicking in one latent space. On the Unitree G1, the policy transfers zero-shot to hardware, with simulation success rates of 97.04% for kicking and 89.88% for dribbling, while real-robot kicking direction error rises from 4.09 to 10.41 degrees.

Humanoid soccer robots have either covered only a few skills or skipped motion priors entirely and moved unnaturally. SoccerBot distills large-scale human soccer motion-capture data into a structured latent motion prior, letting a single skill-conditioned policy learn moving, dribbling, trapping and kicking in one latent space. On the Unitree G1, the policy transfers zero-shot to hardware, with simulation success rates of 97.04% for kicking and 89.88% for dribbling, while real-robot kicking direction error rises from 4.09 to 10.41 degrees.

The work proposes a Bayesian non-parametric skill prior built on a Dirichlet Process Mixture that models temporally extended skills in a structured latent space, enabling adaptive skill discovery without predefining the number of components; integrated into a hierarchical RL framework, the learned prior guides high-level skill selection while a pretrained decoder generates temporally abstracted actions, and experiments on Franka Kitchen, LIBERO-Long, Meta-World, and a real robot show consistent gains in long-horizon manipulation, achieving over 0.8 success rate within 1.5M steps whereas SAC fails to converge even after 5M steps (below 0.1), with an average improvement of 21.8% over a single-Gaussian prior baseline.

The work proposes a Bayesian non-parametric skill prior built on a Dirichlet Process Mixture that models temporally extended skills in a structured latent space, enabling adaptive skill discovery without predefining the number of components; integrated into a hierarchical RL framework, the learned prior guides high-level skill selection while a pretrained decoder generates temporally abstracted actions, and experiments on Franka Kitchen, LIBERO-Long, Meta-World, and a real robot show consistent gains in long-horizon manipulation, achieving over 0.8 success rate within 1.5M steps whereas SAC fails to converge even after 5M steps (below 0.1), with an average improvement of 21.8% over a single-Gaussian prior baseline.