# Long-horizon Robotic Manipulation

When a mobile manipulator is told to put the toys away in the toy box, it usually relies on a task-agnostic semantic map that may lack the attributes, affordances or concepts the instruction needs. Feijoa has a language-model agent query a posed RGB-D scan after the instruction arrives, inspect the returned candidates and rewrite its queries until the reference resolves. On 103 open-ended grounding queries it reaches 94.2% exact match, against 68.0% for the best prebuilt representation, TextMap; on 50 real Spot trials it completes 35 tasks, 17 of them after recovering from a failed tool call or verification check.

When a mobile manipulator is told to put the toys away in the toy box, it usually relies on a task-agnostic semantic map that may lack the attributes, affordances or concepts the instruction needs. Feijoa has a language-model agent query a posed RGB-D scan after the instruction arrives, inspect the returned candidates and rewrite its queries until the reference resolves. On 103 open-ended grounding queries it reaches 94.2% exact match, against 68.0% for the best prebuilt representation, TextMap; on 50 real Spot trials it completes 35 tasks, 17 of them after recovering from a failed tool call or verification check.

The work proposes a Bayesian non-parametric skill prior built on a Dirichlet Process Mixture that models temporally extended skills in a structured latent space, enabling adaptive skill discovery without predefining the number of components; integrated into a hierarchical RL framework, the learned prior guides high-level skill selection while a pretrained decoder generates temporally abstracted actions, and experiments on Franka Kitchen, LIBERO-Long, Meta-World, and a real robot show consistent gains in long-horizon manipulation, achieving over 0.8 success rate within 1.5M steps whereas SAC fails to converge even after 5M steps (below 0.1), with an average improvement of 21.8% over a single-Gaussian prior baseline.

The work proposes a Bayesian non-parametric skill prior built on a Dirichlet Process Mixture that models temporally extended skills in a structured latent space, enabling adaptive skill discovery without predefining the number of components; integrated into a hierarchical RL framework, the learned prior guides high-level skill selection while a pretrained decoder generates temporally abstracted actions, and experiments on Franka Kitchen, LIBERO-Long, Meta-World, and a real robot show consistent gains in long-horizon manipulation, achieving over 0.8 success rate within 1.5M steps whereas SAC fails to converge even after 5M steps (below 0.1), with an average improvement of 21.8% over a single-Gaussian prior baseline.