IWIBench lets large models rewrite a small model's weights: structured output scores zero across all ten editors, while alias edits hit the ceiling
Synopsis
IWIBench turns gradient-free behavioural weight editing into a scored, controlled task: an editor model gets only four tools for reading and writing weights, running the target and inspecting it, and must make ten kinds of semantic change to Gemma 3 4B IT, after which fresh processes compare target success and control retention before and after the patch. Across ten configurations on the same thirty episodes, 6.1-sol at medium reasoning leads with a macro reward of 0.4601 and Astra follows at 0.4356; no editor scores on structured-response defaults, and only 5.6-terra earns positive reward on answer-length control.
Interpretation
A model can now change another model's behaviour by editing its weights without gradients, and IWIBench is the first controlled benchmark that scores the whole attempt. Earlier methods such as ROME, MEMIT and MEND edited particular factual associations, while weight-space programming lacked a controlled evaluation of the full attempt. Ten editor configurations each ran once on the same Gemma 3 4B IT target and the same thirty episodes, giving 300 records; the pristine target answered 104 of 551 target items correctly, with 2,288 baseline-correct controls.
The editor receives the target model, four tools and a fixed budget, and must submit a weight patch rather than repeat instructions at inference time. The tools cover bounded tensor metadata and slice reads, replacement, fill, scale, copy and low-rank additive writes, generation, and teacher-forced numerical inspection, with no gradients, shell or code execution. Each episode allows 64 turns, 128 admitted tool calls and 262,144 target forward tokens; after submission the pristine and patched checkpoints load in separate fresh processes with the same tokenizer, chat template and inference settings.
Capability diverges both between models and between tasks, and does not scale with reasoning effort. 6.1-sol's macro reward rises from 0.4250 at low reasoning to 0.4601 at medium and falls back to 0.4551 at high, while reasoning token consumption goes from 13,631 to 48,456 and then 174,770. Episode by episode, 6.1-sol medium beats low six times, loses six and ties eighteen; high beats medium four times, loses six and ties twenty, while selective factual withholding rises from 2/9 at low to 6/9 at medium and 9/9 at high.
Perspective
Evaluation is fixed to one revision of google/gemma-3-4b-it with text-only inputs and native BF16 weights, and the ten editor configurations share the same thirty episodes, probe seeds and test seeds. The intended readers are researchers and engineers working on behavioural model editing and weight-level intervention; the stated next step is to study how reinforcement learning can foster this experimentation-based adaptive policy and to design rewards that separate locally from globally successful adjustments.
The material describes the frontier models' capability gap as very likely a recently taught ability and explicitly labels that as inference rather than conclusion; on structured-response defaults every editor scores zero, matching the pristine target's zero under the required JSON convention. The authors also state that a malicious model propagating influence into smaller models through weights remains a hypothetical threat, not evidenced by any observed model.