MO-IKE models prompt construction as a constrained Markov decision process via multi-objective RL, raising edit success on Llama-3.2 from 85.0% to 92.0%, paraphrase consistency from 77% to 79%, and retention by 23.0%
Synopsis
The work proposes MO-IKE, a multi-objective reinforcement learning algorithm that formulates prompt construction for in-context knowledge editing as a Constrained Markov Decision Process and trains a dynamic retriever to optimize competing objectives of reliability, generality, and specificity, improving edit success on Llama-3.2 from 85.0% to 92.0%, paraphrase consistency from 77% to 79%, and retention rate by 23.0% over prior RL-based methods.
Figure 1: Contradicting objectives of reliability and specificity in prompt construction in IKE – Increasing the number of RETAIN demonstrations leads to Retention Success but Edit Failure.
arXivInterpretation
MO-IKE formulates prompt construction for in-context knowledge editing as a Constrained Markov Decision Process and trains a dynamic retriever to optimize the competing objectives of reliability, generality, and specificity. Prior RL-based approaches largely optimize a single objective and make decisions over only part of the prompt construction process, overlooking both the balance of objectives and the global organization of demonstrations; MO-IKE treats the prompt as a structured entity and builds it in a globally coherent way under multiple objectives. The abstract states the algorithm is evaluated on Llama-3.2 and reports changes in edit success, paraphrase consistency, and retention relative to prior RL-based methods.
On Llama-3.2, MO-IKE improves edit success (reliability) from 85.0% to 92.0%, paraphrase consistency (generality) from 77% to 79%, and increases retention rate (specificity) by 23.0% compared to prior RL-based methods. These numbers provide a quantitative improvement over prior RL-based methods across three objectives that are often handled separately in knowledge editing. The evidence comes from the Llama-3.2 evaluation reported in the abstract; the text does not give dataset composition, sample size, or statistical uncertainty.
Perspective
The work targets in-context knowledge editing that is training-free and readily applicable to black-box LLMs, aiming for more balanced and globally coherent prompt construction across reliability, generality, and specificity. The results reported in the abstract are limited to Llama-3.2, so the applicable scope should be understood as that model and the evaluated editing setting.
The abstract does not state the evaluation datasets, the number of edit samples, statistical uncertainty, or how the multi-objective trade-off behaves across different models or knowledge types. These are open questions a reader would still watch when judging generalizability.