Command-line agents often report skipped work as success: Receipt reads kernel wait states and cuts false successes from 34 to 9
Synopsis
When LLM agents operate Web services from the command line, they assume every tool call returns within a bounded time with the requested action finished; but many commands ask a question first, and a harness that closes input can let a program return status 0 without doing the work, while an open input makes it wait until a time limit. Receipt keeps input open, reads from system calls what each process waits for, and returns that state to an unmodified agent; across 5 LLMs it lowers false successes from 34 under closed input to 9 and uses 24.4% fewer tokens than a plain terminal.
Interpretation
An agent's reliability depends not only on its LLM but on what its harness lets it observe: whether a command actually performed the requested action when nobody answers is not visible in the tool return. Prior benchmarks scored only the final state and did not record what the process waited for; with closed input a program reads end of file and takes its default, so a skipped action can still return status 0. Among 158 prompting command-line invocations, closed input makes 11.5% return status 0 without performing the requested action, and 7 of 8 harnesses close input when no terminal is attached.
Receipt does not decide for the agent; it returns the process state the kernel exposes so the agent can judge whether a command is still waiting for an answer. Earlier detectors read output text or silence, so a prompt pattern misses a question that never reaches the output and an inactivity watchdog ends a command that is still computing. On 120 locally runnable Litmus invocations that wait under a terminal, Receipt reports 85.0% of the waits with no false positives at a median report delay of 1.1 seconds; plugged into mini-SWE-agent, it raises successes across 5 LLMs from 153 to 176 and lowers false successes from 34 to 9.
In harnesses that keep a persistent terminal, the order of input, interruption and report must be fixed, or the agent's next queued keystroke becomes the answer to a question. Native Terminus 2 returns after the agent's chosen delay, so the next scheduled keystroke answers the question; in the migration task it feeds the queued cd command to the script's question and rebuilds the database. With Receipt, Terminus 2 completes all 25 trap tasks with no false success, against 17 for the native version and 20 for the ablation that returns a single-line interruption notice.
Perspective
The demonstrated scope is measurement over 158 prompting command-line invocations, 91,469 production tool calls and 89,047 public trajectories, plus repair results on the first 15 Tripwire pairs in mini-SWE-agent and all 25 pairs in Terminus 2; the intended reader is an engineer building or evaluating command-line agents. The detector ships as an opt-in feature in a commercial agent runtime, where its checks take a median of 0.11 milliseconds per call on Windows and 0.54 on Linux.
The authors note that findings may not hold for other harnesses and LLMs, so measurement covers public trajectories of 23 harnesses and 19 LLMs and Tripwire runs on two terminal designs; a benchmark written alongside the detector could favor Receipt, so the last 10 of the 25 task pairs were written after the detector was frozen. Of the 18 waits Receipt does not report under a terminal, 15 exit or act without waiting under its launch and 3 are npm commands that draw a spinner while they wait.