Personal memory updates split in two: MemShift scores missed changes apart from wrong repairs

Updating a personal memory splits into two steps: noticing that an input changed, and producing the correct new value. For every changed item, MemShift records whether the memory invalidated it, whether the re-derived value is right, and what the memory served when it did not invalidate. Benchmarks such as LoCoMo, LongMemEval and MemoryAgentBench score only the final answer after new information arrives, so a missed detection and a failed repair look the same; they also state every change as new text, while a redefinition, a writer switch and an export add no text. On the redefinition branch, a memory that never notices the change answers 7.5% of the changed items correctly, and a memory that notices nearly every change and asks its writer to re-derive answers 6.7%; the scores are close, but the two fail at different steps.

Detection is worth only as much as the repair that follows. A keyed memory invalidates an item whose inputs changed and re-derives it, but the invalidation turns into accuracy only when the re-derivation itself is reliable. With Qwen3.5-9B as writer, a key raised Update after a redefinition by only +0.2 percentage points when the writer re-derived the item, by +25.4 points when the writer's own programs re-derived it, and by +26.9 points with exact recomputation. All four open-weight models show the same pattern: for each model the key-by-repair interaction is positive with an interval above zero after a redefinition and after a deletion, ranging from +14.4 to +28.5 points after a redefinition and from +5.0 to +26.4 points after a deletion.

Repair fails mainly on items that aggregate many records under a person's definition. These items require a one-sentence rule to count or collect across calendar, contact and task fields, and a language model re-deriving them tends to compute the wrong value. Each open-weight writer re-derived at most 25% of the changed definition items correctly after every shift, while exact recomputation got at least 91% right; programs the writer writes over typed records close most of that gap. Qwen3.5-9B re-derived only 16.3% of the changed definition items correctly after a source revision, against 99.7% for exact recomputation; on keyed changed definition items, the programs raised Update by +57.4 points over the writer's own re-derivation, against +82.9 points for exact recomputation.

The measurement covers twelve-month synthetic worlds of one person, each with six sources and 48 items, six of which aggregate records under a one-sentence definition; definitions are written over structured fields and gold is computed by a deterministic evaluator. These worlds are synthetic evaluation data, so the results describe synthetic worlds and the tested models, not how a system treats the data of real people.

The worlds are synthetic and template-rendered, and each definition is a one-sentence rule over structured fields, while real records are messier and real definitions vaguer; the main-study intervals rest on 20 worlds, and serving the same store as JSON records changes Update, so the signs of the effects generalize better than their absolute levels.