# Computer Use

Vercel Labs has released agent-desktop, which lets AI agents operate desktop apps on macOS, Windows and Linux without screenshots or pixel matching. It reads an app's accessibility tree as JSON, assigns a ref to every element the agent can act on, and the agent clicks or types on a ref, then reads the UI again to verify. The tool ships as a CLI and a C library, contains no model, and leaves the observe-act-verify loop in the agent.

Vercel Labs has released agent-desktop, which lets AI agents operate desktop apps on macOS, Windows and Linux without screenshots or pixel matching. It reads an app's accessibility tree as JSON, assigns a ref to every element the agent can act on, and the agent clicks or types on a ref, then reads the UI again to verify. The tool ships as a CLI and a C library, contains no model, and leaves the observe-act-verify loop in the agent.

Stablyai released Orca, an open-source app that runs command-line coding agents such as Claude Code and Codex in parallel, giving each agent its own git worktree. Chat and terminal, diff review, a browser, a CLI and phone pairing sit in the same application, which also accepts files or images dragged into an agent's prompt, schedules prompts, and lets agents drive desktop apps when no CLI or API exists. According to the release notes, any CLI agent that runs in a terminal can connect to Orca.

Stablyai released Orca, an open-source app that runs command-line coding agents such as Claude Code and Codex in parallel, giving each agent its own git worktree. Chat and terminal, diff review, a browser, a CLI and phone pairing sit in the same application, which also accepts files or images dragged into an agent's prompt, schedules prompts, and lets agents drive desktop apps when no CLI or API exists. According to the release notes, any CLI agent that runs in a terminal can connect to Orca.

OpenAI released GPT-6.1 Terra, a fast, efficient model combining detailed visual reasoning with precise computer use, priced 50% below GPT-5.6 Terra and GPT-6.1 Sol on the standard API. On Agents' Last Exam at medium reasoning effort it scores 15 percentage points above GPT-5.6 Terra at about one-third the cost, and on Terminal-Bench 4.0 at maximum effort it solves 50.3% of tasks versus GPT-5.6 Terra's published 21.5%. OpenAI notes that GPT-6.1 Sol and GPT-6 Astra remain the better choice for the most complex coding tasks.

OpenAI released GPT-6.1 Terra, a fast, efficient model combining detailed visual reasoning with precise computer use, priced 50% below GPT-5.6 Terra and GPT-6.1 Sol on the standard API. On Agents' Last Exam at medium reasoning effort it scores 15 percentage points above GPT-5.6 Terra at about one-third the cost, and on Terminal-Bench 4.0 at maximum effort it solves 50.3% of tasks versus GPT-5.6 Terra's published 21.5%. OpenAI notes that GPT-6.1 Sol and GPT-6 Astra remain the better choice for the most complex coding tasks.

OpenBot places coding agents such as Codex, Claude Code, Grok CLI, OpenCode, Gemini, Cursor CLI and Cline CLI in one local-first desktop workspace, where each agent keeps its own workspace and conversation while sharing local queues, file transfers, an embedded browser and agent-to-agent messaging. It also offers shared desktop channel chats with one task owner, explicit delegation, and Stop, Resume, Reassign, Archive and Restore controls. The release is a development preview: after first-launch consent, agents run with danger-full-access and approvalPolicy: never, so they can read and modify files, run commands, use the network and control the browser without per-action confirmations.

OpenBot places coding agents such as Codex, Claude Code, Grok CLI, OpenCode, Gemini, Cursor CLI and Cline CLI in one local-first desktop workspace, where each agent keeps its own workspace and conversation while sharing local queues, file transfers, an embedded browser and agent-to-agent messaging. It also offers shared desktop channel chats with one task owner, explicit delegation, and Stop, Resume, Reassign, Archive and Restore controls. The release is a development preview: after first-launch consent, agents run with danger-full-access and approvalPolicy: never, so they can read and modify files, run commands, use the network and control the browser without per-action confirmations.

The authors introduce CUA-SWE, a benchmark, interactive development environment, and evaluation pipeline spanning 105 tasks across Web, Game, DevOps, and Mobile, in which an agent edits code, runs commands, operates the running software, and inspects screenshots within a single task, with task-specific deterministic tests deciding the outcome; comparing code-only and hybrid CUA conditions across frontier models, GPT-6-Astra reaches the highest four-domain mean of 59.9%, and every frontier model in the four-domain comparison achieves higher aggregate success with hybrid access, with gains concentrated on tasks whose requirements must be recovered from application materials such as drawings, service contracts, or graphical reference cards.

The authors introduce CUA-SWE, a benchmark, interactive development environment, and evaluation pipeline spanning 105 tasks across Web, Game, DevOps, and Mobile, in which an agent edits code, runs commands, operates the running software, and inspects screenshots within a single task, with task-specific deterministic tests deciding the outcome; comparing code-only and hybrid CUA conditions across frontier models, GPT-6-Astra reaches the highest four-domain mean of 59.9%, and every frontier model in the four-domain comparison achieves higher aggregate success with hybrid access, with gains concentrated on tasks whose requirements must be recovered from application materials such as drawings, service contracts, or graphical reference cards.

The work builds CaptchaArena, a large-scale fine-grained computer-use training dataset of 20,000 interactive CAPTCHA puzzles across 20 types and five interaction modes, where every solution is replayed in a real browser and accepted by the page's own verifier, together with 20,000 screenshot-action trajectories (18,000 carrying judge-filtered step-by-step reasoning annotations) and pixel-mask supervision for irregular targets; training a single 9B policy, CaptchaAgent, on it reaches 70.5 average Pass@1 after supervised fine-tuning and 71.7 after reinforcement learning with the environment verifier as reward, versus 11.4 for the untrained backbone, 35.2 for the strongest open-weight GUI agent, 69.2 for the strongest closed-source model, and 94.1 for humans.

The work builds CaptchaArena, a large-scale fine-grained computer-use training dataset of 20,000 interactive CAPTCHA puzzles across 20 types and five interaction modes, where every solution is replayed in a real browser and accepted by the page's own verifier, together with 20,000 screenshot-action trajectories (18,000 carrying judge-filtered step-by-step reasoning annotations) and pixel-mask supervision for irregular targets; training a single 9B policy, CaptchaAgent, on it reaches 70.5 average Pass@1 after supervised fine-tuning and 71.7 after reinforcement learning with the environment verifier as reward, versus 11.4 for the untrained backbone, 35.2 for the strongest open-weight GUI agent, 69.2 for the strongest closed-source model, and 94.1 for humans.

The announcement introduces GPT-6.1 Sol, described as offering near-Astra intelligence for coding, computer use, and professional work at one-fifth of Astra's standard API input and output token prices.

The announcement introduces GPT-6.1 Sol, described as offering near-Astra intelligence for coding, computer use, and professional work at one-fifth of Astra's standard API input and output token prices.