Cheng Yang

AutoGUIWorld introduces a data generation framework that combines the visual priors of image generators with the task knowledge of a planner to synthesize GUI interaction trajectories without deploying or running the corresponding software environments: it samples initial GUI scenes from structured specifications of operating-system context, visual appearance, and interface state, generates tasks conditioned on those scenes, has a planner specify atomic actions and their intended visual consequences, and has an image generator iteratively edit the current screenshot to produce subsequent observations, yielding 79,266 spatially annotated step-level training samples across Ubuntu, Windows, macOS, and Chrome; fine-tuning Qwen3.

AutoGUIWorld introduces a data generation framework that combines the visual priors of image generators with the task knowledge of a planner to synthesize GUI interaction trajectories without deploying or running the corresponding software environments: it samples initial GUI scenes from structured specifications of operating-system context, visual appearance, and interface state, generates tasks conditioned on those scenes, has a planner specify atomic actions and their intended visual consequences, and has an image generator iteratively edit the current screenshot to produce subsequent observations, yielding 79,266 spatially annotated step-level training samples across Ubuntu, Windows, macOS, and Chrome; fine-tuning Qwen3.