# Computer-Use Agents

Computer-use agents encode each screenshot into thousands of visual tokens, and those tokens pile up into long contexts on long-horizon tasks and in real-time games. VTR trains a vision-language model jointly with a lightweight spatio-temporal router that inspects the same image region in the current and previous screenshots and decides whether that token enters the language model, using an adaptive loss to keep as few tokens as possible without losing action prediction. After only 2,000 training steps, VTR matches the full-resolution model on OSWorld, CUA-World and video-game action prediction while using 2.5–5.8× fewer visual tokens, cutting inference cost by up to 3.3× and speeding prefill by up to 2.9×.

Computer-use agents encode each screenshot into thousands of visual tokens, and those tokens pile up into long contexts on long-horizon tasks and in real-time games. VTR trains a vision-language model jointly with a lightweight spatio-temporal router that inspects the same image region in the current and previous screenshots and decides whether that token enters the language model, using an adaptive loss to keep as few tokens as possible without losing action prediction. After only 2,000 training steps, VTR matches the full-resolution model on OSWorld, CUA-World and video-game action prediction while using 2.5–5.8× fewer visual tokens, cutting inference cost by up to 3.3× and speeding prefill by up to 2.9×.

Computer-use agents that act directly on graphical interfaces must branch on runtime web content, so an adversary can craft untrusted data that pushes the agent into a hazardous pre-approved branch without injecting explicit instructions. STEER-Bench measures this branch steering across 101 tasks in 9 domains, with 94.4% attack success against standard agents and 89.5% against vanilla Dual-LLM. COBRA pairs trusted branching plans with ahead-of-time capability constraints that bound each branch's parameters and destinations, cutting attack success to 0% while retaining 97% benign utility.

Computer-use agents that act directly on graphical interfaces must branch on runtime web content, so an adversary can craft untrusted data that pushes the agent into a hazardous pre-approved branch without injecting explicit instructions. STEER-Bench measures this branch steering across 101 tasks in 9 domains, with 94.4% attack success against standard agents and 89.5% against vanilla Dual-LLM. COBRA pairs trusted branching plans with ahead-of-time capability constraints that bound each branch's parameters and destinations, cutting attack success to 0% while retaining 97% benign utility.