W. J. Chu

Long-running LLM agents compact their context by writing a handoff summary that the next instance continues from, and that summary can state permissions or requirements the record does not support. Using synthetic tool-use tasks with a ground-truth permission ledger and same-snapshot replay, two AI readers confirmed unsupported restrictions in 1.2% and 2.4% of readable summaries from two open-weight models on held-out instances, and unsupported authorizations in 0.1% and 1.3%. Replacing the system prompt's "Some actions need approval." with a sentence saying where requirements come from cut unsupported restrictions from 5.0% to 1.8% of development summaries and from 1.8% to 0.3% on held-out templates, while requests for approvals the ledger requires barely changed.

Long-running LLM agents compact their context by writing a handoff summary that the next instance continues from, and that summary can state permissions or requirements the record does not support. Using synthetic tool-use tasks with a ground-truth permission ledger and same-snapshot replay, two AI readers confirmed unsupported restrictions in 1.2% and 2.4% of readable summaries from two open-weight models on held-out instances, and unsupported authorizations in 0.1% and 1.3%. Replacing the system prompt's "Some actions need approval." with a sentence saying where requirements come from cut unsupported restrictions from 5.0% to 1.8% of development summaries and from 1.8% to 0.3% on held-out templates, while requests for approvals the ledger requires barely changed.