# Knowledge Editing

EngramEdit proposes decoupled knowledge updates through conditional memory: it first computes target memory representations that make the model predict an updated fact across multiple expressions, then jointly solves updates to shared n-gram embeddings while penalizing changes to frequently reused embeddings more strongly; on LongCat-Flash-Lite, updating only conditional memory yields near-perfect editing success on CounterFact and ZsRE, revised facts transfer to unseen expressions and multi-hop reasoning, and unrelated knowledge and general capabilities are largely preserved.

EngramEdit proposes decoupled knowledge updates through conditional memory: it first computes target memory representations that make the model predict an updated fact across multiple expressions, then jointly solves updates to shared n-gram embeddings while penalizing changes to frequently reused embeddings more strongly; on LongCat-Flash-Lite, updating only conditional memory yields near-perfect editing success on CounterFact and ZsRE, revised facts transfer to unseen expressions and multi-hop reasoning, and unrelated knowledge and general capabilities are largely preserved.

The work proposes MO-IKE, a multi-objective reinforcement learning algorithm that formulates prompt construction for in-context knowledge editing as a Constrained Markov Decision Process and trains a dynamic retriever to optimize competing objectives of reliability, generality, and specificity, improving edit success on Llama-3.2 from 85.0% to 92.0%, paraphrase consistency from 77% to 79%, and retention rate by 23.0% over prior RL-based methods.

The work proposes MO-IKE, a multi-objective reinforcement learning algorithm that formulates prompt construction for in-context knowledge editing as a Constrained Markov Decision Process and trains a dynamic retriever to optimize competing objectives of reliability, generality, and specificity, improving edit success on Llama-3.2 from 85.0% to 92.0%, paraphrase consistency from 77% to 79%, and retention rate by 23.0% over prior RL-based methods.

The work identifies context reliance in unstructured knowledge editing, where the standard passage-level objective gives later facts richer ground-truth context and thus lower initial losses so they appear easier to learn; it proposes FOVEATED, a plug-and-play framework that constructs focused views of each sentence by randomly shifting the RoPE positions assigned to the keys of its preceding context during editing and removing the perturbation afterward so native positional encoding is unchanged at inference, instantiated for both direct-optimization and locate-then-edit editors, with a theoretical analysis of how it counteracts context-induced difficulty underestimation and empirical improvements across five KE editors, two LLM backbones, and three benchmarks.

The work identifies context reliance in unstructured knowledge editing, where the standard passage-level objective gives later facts richer ground-truth context and thus lower initial losses so they appear easier to learn; it proposes FOVEATED, a plug-and-play framework that constructs focused views of each sentence by randomly shifting the RoPE positions assigned to the keys of its preceding context during editing and removing the perturbation afterward so native positional encoding is unchanged at inference, instantiated for both direct-optimization and locate-then-edit editors, with a theoretical analysis of how it counteracts context-induced difficulty underestimation and empirical improvements across five KE editors, two LLM backbones, and three benchmarks.

This Perspective addresses LLM agents that accumulate state across sessions, tools, users, and changing environments, and proposes a framework of controlled knowledge updating: treating an update as a decision to select the least invasive substrate sufficient for the claim's scope—among context, external memory, model parameters, activation states, and tool or workflow definitions—including the option not to write persistent state, organized by three principles (substrate proportionality, temporal defeasibility, auditable continuity) and motivating evaluation criteria covering update selection, temporal consistency, interference, reversibility, efficiency, and robustness, together with a four-phase minimal benchmark unit.

This Perspective addresses LLM agents that accumulate state across sessions, tools, users, and changing environments, and proposes a framework of controlled knowledge updating: treating an update as a decision to select the least invasive substrate sufficient for the claim's scope—among context, external memory, model parameters, activation states, and tool or workflow definitions—including the option not to write persistent state, organized by three principles (substrate proportionality, temporal defeasibility, auditable continuity) and motivating evaluation criteria covering update selection, temporal consistency, interference, reversibility, efficiency, and robustness, together with a four-phase minimal benchmark unit.