# VLM-Guided Gaze Targeting

Gaze control for a humanoid face robot is split in two: a vision-language model asynchronously picks the scene referent, and a Realistic Gaze Transformer turns a target eye trajectory into coordinated neck, binocular-eye, and eyelid motion. The model is a Transformer-VAE trained on markerless human recordings, first self-supervised and then autoregressively fine-tuned so training includes its own predictions. On a two-second closed-loop benchmark where every controller receives the same ground-truth eye target, RGT-Base reaches the lowest reported seven-joint error, with inference inside the 16.7 ms frame interval.

Gaze control for a humanoid face robot is split in two: a vision-language model asynchronously picks the scene referent, and a Realistic Gaze Transformer turns a target eye trajectory into coordinated neck, binocular-eye, and eyelid motion. The model is a Transformer-VAE trained on markerless human recordings, first self-supervised and then autoregressively fine-tuned so training includes its own predictions. On a two-second closed-loop benchmark where every controller receives the same ground-truth eye target, RGT-Base reaches the lowest reported seven-joint error, with inference inside the 16.7 ms frame interval.