# GUI Agent

For GUI agents, the safety signal may lie not in the screen but in the transition that produced it: the same credential screen can be a legitimate result of tapping Sign in or a hijacked result of tapping View order. LGWM skips rendering the next screen and predicts its representation directly in the observation encoder's space, reducing verification to a vector comparison; on RSWT-Bench, where each credential screen appears pixel-identically under both labels, the training-free consistency score reaches 0.987 AUC at 17 ms per decision, on par with the strongest closed-source vision-language models and about ten AUC points above generative GUI world models.

For GUI agents, the safety signal may lie not in the screen but in the transition that produced it: the same credential screen can be a legitimate result of tapping Sign in or a hijacked result of tapping View order. LGWM skips rendering the next screen and predicts its representation directly in the observation encoder's space, reducing verification to a vector comparison; on RSWT-Bench, where each credential screen appears pixel-identically under both labels, the training-free consistency score reaches 0.987 AUC at 17 ms per decision, on par with the strongest closed-source vision-language models and about ten AUC points above generative GUI world models.

GUI agents no longer reconstruct the next screen pixel by pixel; a shared latent representation now scores candidate actions and predicts action-conditioned futures at once, raising success rates by up to 31.9% offline and 40.6% online relative to pixel-space world models while cutting world-model inference latency by up to 98.9%.

GUI agents no longer reconstruct the next screen pixel by pixel; a shared latent representation now scores candidate actions and predicts action-conditioned futures at once, raising success rates by up to 31.9% offline and 40.6% online relative to pixel-space world models while cutting world-model inference latency by up to 98.9%.

Fine-tuning Qwen2-VL-2B with LoRA on Android in the Wild, the study compares five ways of supplying the action type to a small GUI grounding model under matched data, compute, and decoding, finding that an auxiliary loss, an additive embedding, and a prompt word each raise hit@0.10 by five to seven points, but after retraining without the degenerate class created by the serializer no mechanism beats the baseline on hit rate, and only the auxiliary loss shortens the average miss.

Fine-tuning Qwen2-VL-2B with LoRA on Android in the Wild, the study compares five ways of supplying the action type to a small GUI grounding model under matched data, compute, and decoding, finding that an auxiliary loss, an additive embedding, and a prompt word each raise hit@0.10 by five to seven points, but after retraining without the degenerate class created by the serializer no mechanism beats the baseline on hit rate, and only the auxiliary loss shortens the average miss.