Public articles linked to the same research event.
arXiv The work identifies context reliance in unstructured knowledge editing, where the standard passage-level objective gives later facts richer ground-truth context and thus lower initial losses so they appear easier to learn; it proposes FOVEATED, a plug-and-play framework that constructs focused views of each sentence by randomly shifting the RoPE positions assigned to the keys of its preceding context during editing and removing the perturbation afterward so native positional encoding is unchanged at inference, instantiated for both direct-optimization and locate-then-edit editors, with a theoretical analysis of how it counteracts context-induced difficulty underestimation and empirical improvements across five KE editors, two LLM backbones, and three benchmarks.
The work identifies context reliance in unstructured knowledge editing, where the standard passage-level objective gives later facts richer ground-truth context and thus lower initial losses so they appear easier to learn; it proposes FOVEATED, a plug-and-play framework that constructs focused views of each sentence by randomly shifting the RoPE positions assigned to the keys of its preceding context during editing and removing the perturbation afterward so native positional encoding is unchanged at inference, instantiated for both direct-optimization and locate-then-edit editors, with a theoretical analysis of how it counteracts context-induced difficulty underestimation and empirical improvements across five KE editors, two LLM backbones, and three benchmarks.
The work identifies context reliance in unstructured knowledge editing, where the standard passage-level objective gives later facts richer ground-truth context and thus lower initial losses so they appear easier to learn; it proposes FOVEATED, a plug-and-play framework that constructs focused views of each sentence by randomly shifting the RoPE positions assigned to the keys of its preceding context during editing and removing the perturbation afterward so native positional encoding is unchanged at inference, instantiated for both direct-optimization and locate-then-edit editors, with a theoretical analysis of how it counteracts context-induced difficulty underestimation and empirical improvements across five KE editors, two LLM backbones, and three benchmarks.
The work identifies context reliance in unstructured knowledge editing, where the standard passage-level objective gives later facts richer ground-truth context and thus lower initial losses so they appear easier to learn; it proposes FOVEATED, a plug-and-play framework that constructs focused views of each sentence by randomly shifting the RoPE positions assigned to the keys of its preceding context during editing and removing the perturbation afterward so native positional encoding is unchanged at inference, instantiated for both direct-optimization and locate-then-edit editors, with a theoretical analysis of how it counteracts context-induced difficulty underestimation and empirical improvements across five KE editors, two LLM backbones, and three benchmarks.