A big model's evolved robot harness still works at 4B: 62.8% success after distillation
Synopsis
TinyAstral moves the improvement target outside the robot agent's planner: an open-weight teacher is frozen while frontier LLM agents read its trajectories and propose harness edits, each accepted only if it passes a paired statistical test, lifting the teacher's strict RoboTwin success from 33.4% to 63.6%. The final harness is then frozen and the teacher's reasoning and failure-recovery trajectories are distilled into a 4B student, which reaches 62.8% in that same harness and tracks its corresponding teacher across four harness versions.
Interpretation
A robot manipulation agent's competence depends not only on its planner model but on the harness around it, meaning the tool interface, execution feedback, task memory and failure-recovery logic; TinyAstral improves that layer while the teacher's weights stay frozen throughout. Harness evolution and agent distillation had been separate lines: one improved the system around a frozen planner, the other trained a small model from a stronger agent's trajectories, and the two had not been joined into one pipeline. On 500 RoboTwin episodes covering all 50 tasks and 10 evaluation seeds, the teacher Qwen3.8-Flash-Next rises from 33.4% strict success under the starting harness to 63.6% under the final one, with unchanged weights.
The harness gains are not tied to one planner; other frozen models benefit as well. DeepSeek-V4.1-Flash, which did not take part in harness evolution, improves by 30.6 percentage points across the same harness versions, and the untrained 4B planner rises from 14.6% to 41.2%, with all three improving across the four measured versions. All three planners are evaluated on the same four harness versions and the same 500 RoboTwin episodes, without model-specific harness changes.
Distillation carries the harness gains into small models: one 4B student is trained per harness version, and student scores rise with the version and stay close to their corresponding teachers. The four students reach 36.2%, 49.8%, 60.0% and 62.8% strict success in order, and under the final harness the student beats the untrained 4B planner by 21.6 percentage points. Each student is trained independently from the same Qwen3.5-4B checkpoint with the same recipe on successful episodes collected under its own harness version, and evaluated in that harness.
The final 4B student outperforms the open competitors run here on identical episodes and comes close to a 552B planner on three benchmarks never used for evolution or training. On Guava the student scores 93.6% against 78.9% for its own teacher placed in the Guava harness; on LIBERO-PRO it scores 78.3% against 57.3% and 53.0% for the same teacher in the MotorMind and PyRUA-Lean harnesses. On SimplerEnv, VLABench and EB-Manipulation the student stays within 5.0 percentage points of its teacher and within 2.4 points of the 552B DeepSeek-V4.1-Flash in the same harness; in the deployment measurement it uses about one fifth of the teacher's accelerator time per episode.
Perspective
The demonstrated scope is manipulation tasks on simulated benchmarks: the three training benchmarks RoboTwin, Guava and LIBERO-PRO, plus SimplerEnv, VLABench and EB-Manipulation, which are never used for harness evolution or training. On real hardware the same student sorts two balls into matching boxes and stacks three tape rolls on an SO-ARM101 arm. The compute comparison comes from a deployment measurement on 20 SimplerEnv episodes. The material states that future work will extend the approach to more RSI environments.
The material limits the transfer of harness gains to the planners and harness versions tested, and reports that model size matters most on LIBERO-PRO: the 0.8B student reaches 61.8% on RoboTwin and 93.8% on Guava but falls to 54.7% on LIBERO-PRO, where the 4B student reaches 78.3%. Real-robot evidence is confined to two demonstration tasks on an SO-ARM101 arm.