Robot memory stops guessing from keyframes: per-tick text lines lift success from 44.5% to 66.0% under a fixed policy

Robot memory tasks no longer depend on picking keyframes from a whole video. The best prior non-privileged method, FrameSamp+Modul, averaged 44.5% success across RoboMME's sixteen tasks against a privileged subgoal ceiling of 84.1%. Gamma raises the average to 66.0% with the same fixed execution policy, matching that ceiling on PickXtimes, PatternLock and RouteStick. Earlier systems left memory to a single vision-language model that selected keyframes and reasoned over them: MemER averaged 42.4% and GroundSG+QwenVL 32.7%, both far below the privileged ceiling. The comparison uses RoboMME's sixteen memory tasks, 30 episodes per task per serving seed over three seeds, with the executor's weights identical to the privileged rows and only the subgoal stream differing.

The system distils each replan window into at most one grounded text line, appends it to a memory bank that is never rewritten, and has a second model read only that text, with no pixels, to emit the next subgoal. Keyframe memory could drop the decisive frame with no way to recover it, and coordinates came straight from a vision-language model, which often landed on the wrong one of several identical objects. SAM-3 supplies per-frame coordinates from class text prompts; the writer and reasoner share a Qwen3.5-9B base with separate LoRA adapters, and the corpus is generated mechanically from 1,600 recorded episodes under the same evidence gates.

Every claim an agent makes must pass a mechanical check first: event lines, completion claims and cited coordinates are each tested against the window's detection evidence and admitted, deferred, corrected or rejected. Without that check the writer declared invisible actions complete on its learned schedule, and both agents cited coordinates they could not know. Removing the defer verdict drops ButtonUnmask from 84.4% to 56.7% and ButtonUnmaskSwap from 35.6% to 13.1%; removing reject drops PatternLock and RouteStick from 98.9% and 56.7% to 3.3% and 6.7%.

All results are simulation: RoboMME's sixteen memory tasks and RoboMemArena's 26 kitchen tasks, with the execution policy held fixed and a privileged ceiling plus 480 held-out episodes per configuration making the comparison controlled. It suits readers working on long-horizon robot memory, planning above vision-language-action policies, and verification of agent output. The authors note that transferring to a new environment requires re-deriving such readers or a vision-language model that can genuinely read motion, which they call the open research problem.

The authors state this is simulation only and not a claim about physical manipulation, which enters only through the fixed executor and the detector. Each tick costs one detector pass and two model generations, and the closed-loop runs wait for the call, so the reported success rates do not depend on its latency. The two direction tasks rely on deterministic readers that encode benchmark regularities such as fixed stroke length and marker semantics, which the authors treat as the current boundary of vision-language models.