Justus Flerlage

Instead of drawing on a fixed command library, a quadruped exploring unknown space hands the user's high-level intention to a large language model that writes an executable program restricted to a robot function library, while a vision-language model supplies open-ended semantic labels and geometric and semantic thresholds trigger replanning. In Gazebo simulation, Qwen3.5 and Kimi-K2.5 reach about 82% overall success and GPT-5-nano about 46%; finite tasks hold up well, while open-ended tasks vary more.

Instead of drawing on a fixed command library, a quadruped exploring unknown space hands the user's high-level intention to a large language model that writes an executable program restricted to a robot function library, while a vision-language model supplies open-ended semantic labels and geometric and semantic thresholds trigger replanning. In Gazebo simulation, Qwen3.5 and Kimi-K2.5 reach about 82% overall success and GPT-5-nano about 46%; finite tasks hold up well, while open-ended tasks vary more.

The work introduces a MAPE-K-inspired dual-AI architecture in which an LLM translates high-level natural-language intentions into executable Python code constrained to a robotic function library, while a VLM provides open-ended semantic grounding through a distillation process, with geometric and semantic thresholds triggering replanning; benchmarked on quadruped exploration tasks in Gazebo across three frontier models, finite tasks show high success rates while open-ended exhaustive tasks degrade, with failures concentrated in semantic grounding, completeness checking, and termination control rather than low-level action execution.

The work introduces a MAPE-K-inspired dual-AI architecture in which an LLM translates high-level natural-language intentions into executable Python code constrained to a robotic function library, while a VLM provides open-ended semantic grounding through a distillation process, with geometric and semantic thresholds triggering replanning; benchmarked on quadruped exploration tasks in Gazebo across three frontier models, finite tasks show high success rates while open-ended exhaustive tasks degrade, with failures concentrated in semantic grounding, completeness checking, and termination control rather than low-level action execution.