Delivered memory is not used memory: MemUse traces each delivery and finds most entries irrelevant and most relevant ones ignored

The object of memory evaluation shifts from final task success to each memory delivery: once an entry enters a model input, whether the agent used it and what followed can now be judged on its own. Earlier memory benchmarks mainly compared overall task performance with and without memory, or relied on gold answers, reference operation traces and controlled memory interventions, so a single recorded run could not show what one delivery did. Over all 500 SWE-bench Verified tasks and four configurations of three memory systems, giving 8,000 memory episodes and 2,000 baseline episodes, resolve rates ranged from 72.65% to 73.35% against 72.95% without memory, and every 95% interval for the difference contained zero.

Each delivery is scored separately on information quality, use and consequence, so relevant-but-unused and used-but-harmful no longer collapse into one task result. An agentic judge receives the delivered text, the context the agent could already see, and the window of later actions the entry can claim, and must cite trajectory steps it has read in full. Programmatic checks enforce complete reads of cited evidence, and fixed rules aggregate the ratings into span verdicts; of 15,271 judged spans, 949 triggered follow-up reads and all ultimately passed validation.

The demonstrated scope is recorded trajectories of mini-SWE-agent on the 500 Python tasks of SWE-bench Verified, covering MemGovern, two Mem0 configurations and TencentDB Agent Memory; verdicts describe observed behavior in recorded episodes, not counterfactual effects on task success.

Agreement between the judge and human judgments has not been measured in this evaluation, and overlap between MemGovern's corpus and the benchmark problems' own discussions and repairs is established for specific cases while its prevalence across the full corpus remains unmeasured.