# Context Compaction

Long-running LLM agents compact their context by writing a handoff summary that the next instance continues from, and that summary can state permissions or requirements the record does not support. Using synthetic tool-use tasks with a ground-truth permission ledger and same-snapshot replay, two AI readers confirmed unsupported restrictions in 1.2% and 2.4% of readable summaries from two open-weight models on held-out instances, and unsupported authorizations in 0.1% and 1.3%. Replacing the system prompt's "Some actions need approval." with a sentence saying where requirements come from cut unsupported restrictions from 5.0% to 1.8% of development summaries and from 1.8% to 0.3% on held-out templates, while requests for approvals the ledger requires barely changed.

Long-running LLM agents compact their context by writing a handoff summary that the next instance continues from, and that summary can state permissions or requirements the record does not support. Using synthetic tool-use tasks with a ground-truth permission ledger and same-snapshot replay, two AI readers confirmed unsupported restrictions in 1.2% and 2.4% of readable summaries from two open-weight models on held-out instances, and unsupported authorizations in 0.1% and 1.3%. Replacing the system prompt's "Some actions need approval." with a sentence saying where requirements come from cut unsupported restrictions from 5.0% to 1.8% of development summaries and from 1.8% to 0.3% on held-out templates, while requests for approvals the ledger requires barely changed.

Long-horizon language-model agents accumulate ever-longer interaction histories, so context compaction is necessary, yet existing learned compaction methods often depend on downstream task outcomes or solver behavior. SumOne is a standalone 2-billion-parameter summarizer trained with a task-agnostic reward built from masked reconstruction and optimized with GRPO, using only raw trajectories and no reference summaries or downstream task execution. On DeepSearchQA it cuts cumulative agent input from 2.83M to 591K tokens with F1 0.48 versus 0.52 for full context; on SWE-bench Verified it reaches a 0.59 resolved rate versus 0.63, with input falling from 4.10M to 1.09M tokens.

Long-horizon language-model agents accumulate ever-longer interaction histories, so context compaction is necessary, yet existing learned compaction methods often depend on downstream task outcomes or solver behavior. SumOne is a standalone 2-billion-parameter summarizer trained with a task-agnostic reward built from masked reconstruction and optimized with GRPO, using only raw trajectories and no reference summaries or downstream task execution. On DeepSearchQA it cuts cumulative agent input from 2.83M to 591K tokens with F1 0.48 versus 0.52 for full context; on SWE-bench Verified it reaches a 0.59 resolved rate versus 0.63, with input falling from 4.10M to 1.09M tokens.

Coding agents mostly append to their history, so prefix-cache reuse reaches 98–99%, yet text-level context compaction rewrites earlier tokens and forces the whole suffix to be recomputed. ObsDrop moves compaction from the prompt into the KV cache: it keeps the logical token prefix unchanged while releasing the KV of stale tool observations and excluding those positions from attention, implemented in SGLang and Sparse-vLLM. At 16 concurrent agents it reaches 2.45–2.85× throughput with FlashInfer and a 5.37× peak with Triton relative to no compaction, with task quality close to no compaction.

Coding agents mostly append to their history, so prefix-cache reuse reaches 98–99%, yet text-level context compaction rewrites earlier tokens and forces the whole suffix to be recomputed. ObsDrop moves compaction from the prompt into the KV cache: it keeps the logical token prefix unchanged while releasing the KV of stale tool observations and excluding those positions from attention, implemented in SGLang and Sparse-vLLM. At 16 concurrent agents it reaches 2.45–2.85× throughput with FlashInfer and a 5.37× peak with Triton relative to no compaction, with task quality close to no compaction.

Remory trains a neural memory network to append a bounded sequence of soft memory tokens after a generated summary, helping a frozen LLM approximate the continuation it would produce with the full history; on SummHay it raises citation F1 by 4.04 points and joint score by 3.55 using about 5.2% of input positions, and yields consistent gains for Qwen3.8-27B and GLM-5.3-Flash on long-horizon agent benchmarks while reducing repeated tool outputs and tool errors.

Remory trains a neural memory network to append a bounded sequence of soft memory tokens after a generated summary, helping a frozen LLM approximate the continuation it would produce with the full history; on SummHay it raises citation F1 by 4.04 points and joint score by 3.55 using about 5.2% of input positions, and yields consistent gains for Qwen3.8-27B and GLM-5.3-Flash on long-horizon agent benchmarks while reducing repeated tool outputs and tool errors.