Skip to main content
Back to timeline
researchsquare.comSource publication:

Recorded GUI operations become agent tools: 33/33 core points on three SaaS instances, against 8, 8 and 18 for raw code

Synopsis

Recording real graphical-interface operations, parameterizing them into a frozen workflow and handing that to an agent can cover more business requirements than letting the agent write code directly. On three complete-task instances of one SaaS business family, the frozen workflow reached 33/33 local core points while raw code reached 8, 8 and 18; on two prospectively fixed instances the tool agents finished in six and five model calls, against 121 and 70 for raw code. Every attempt still failed whole-task acceptance, and collateral state preservation remains unknown.

Source-provided article image: Goal Completion and State Preservation: Case Studies of GUI-Derived Agent Tools
researchsquare.com

Interpretation

A workflow recorded from real graphical-interface operations, parameterized and then frozen, covers more checks than an agent writing code directly on complete business requests spanning Twenty CRM, BigCapital and Pretix. The raw-code agent reached only 8/33, 8/33 and 18/33 local core points on the same tasks, with 0/20, 0/20 and 6/20 additional checks. The frozen workflow scored 33/33 core points and 16/20 additional checks on all three instances; the original evaluator gives the same ordering, 30/33 versus 11/33, 11/33 and 21/33.

The workflow expands one business goal into a fixed sequence of native calls, and the agent only fills in goal parameters. Recording retains the request method, route template, payload structure and identifier-extraction paths of 77 operations, while new business names, amounts and dates become goal parameters. The execution agent maps a natural-language request to a two-milestone goal and submits parameters through a protected interface, which expands the macro in catalog order; scoring uses read-only SQL on terminal containers to read 18 core checks worth 33 weighted points plus 20 additional checks.

Two prospectively fixed parameter instances reproduce the coverage advantage within the same business family with far fewer model decisions. DEV009 and DEV010 were fixed after DEV008 and run once per slot with the workflow implementation unchanged. The two new tool agents finished normally after six and five calls, using 106,848 and 73,180 execution tokens; paired raw code hit the scientific token limit in DEV009 after its 121st model response and finished normally in DEV010 after 70 calls.

Perspective

The conclusions cover three complete-task instances of one SaaS business family and two Magento base tasks, using one requested executor model and reasoning setting. DEV008 is a development comparison, while DEV009 and DEV010 were prospectively fixed and run once per arm. The Magento results hold within its declared state boundary and native probes; image rendering, live customer-group-1 transactions and arbitrary concurrency lie outside them.

Between raw code and tools, workflow knowledge, operation implementation and execution restrictions change together, so the comparison identifies their combined effect. Static and guarded modes tie where both are evaluated, Magento's learned predicate overlaps shared validation, and the original repository's negative-price response was not independently established. SaaS collateral preservation is unmeasured, and the description-propagation assay measures a conditional mechanism rather than a prospective error rate. Actual GUI attempts end on interruption and invalid actions, limiting comparison with normally completed execution.