Web-GUI agents pay for every screenshot pixel: keeping 21% of visual tokens cuts latency 31% and holds 77.7% of performance

A web-GUI agent can run on roughly one fifth of its visual tokens and still complete most of the grounding work that full screenshot tokens supported. Earlier token pruning methods, designed mainly for natural images, selected tokens independently by importance score, which can fragment the local visual structures needed to recognize small GUI elements such as buttons, icons, and hyperlinks. On Online-Mind2Web, retaining 21% of tokens preserves 77.7% of full-token performance while reducing inference latency by 31% and FLOPs by 57%.

Token choice no longer rests on single-token importance alone; instruction-conditioned scores are computed first, then tokens are kept or dropped in spatially coherent windows. The approach is query-guided token pruning: it preserves fine-grained instruction-conditioned token scoring while performing spatially coherent window-level selection, so aggressive reduction does not shatter local structure. A lightweight selector handles the choice and is optimized through policy learning while the VLM backbone remains frozen; evaluation runs on Online-Mind2Web, and results on Android in the Wild suggest transfer to mobile GUI environments.

This report covers summary-level information: the method reports an efficiency-performance tradeoff on Online-Mind2Web at 21% token retention, plus a signal of transfer to mobile GUI environments on Android in the Wild. It suits researchers and engineers tracking inference cost in vision-language models and GUI agent deployment. The summary states no next-step experiment plan.

The summary reports results only under a low token budget and does not describe behavior at higher retention rates or other budget levels; the transfer conclusion is presented as a signal from Android in the Wild rather than full validation. This report is based on the summary and does not verify full-text details.