A paraphrase shouldn't flip a refusal: attribution consistency cuts a small tool agent's flip rate from 11.3% to 9.3%
Synopsis
Tool-using language model agents can refuse one phrasing of a harmful request and comply with another, and a missed refusal becomes an action. Llama-3.2-1B-Instruct was fine-tuned with LoRA on data not drawn from AgentHarm, comparing latent attribution regularization (LAR) against an augmentation-only control over three seeds. At 10% over-refusal on harmless tool tasks, LAR lowered the share of tasks with mixed refusal decisions from 25% to 19% and paraphrase-induced flips from 11.3% to 9.3%, while harmful-versus-harmless detection did not change significantly.
Interpretation
Refusal training is brittle with respect to wording: reformulating a request can recover compliance from a model that otherwise refuses, and for a tool-using agent a missed refusal becomes an executed action. Earlier diagnoses mostly concern chat-style prompts, whereas the decision here is between refusing and acting inside an agent, and too-rare refusal must be separated from inconsistent refusal. Evaluation uses AgentHarm's 176 harmful tasks with three approved paraphrases each, giving 704 harmful prompts, plus 176 harmless tool tasks.
Once over-refusal is matched, the LAR-trained model is less sensitive to paraphrase, so different phrasings of one task more often receive the same decision. The control applies the same paraphrase augmentation, with identical data, seeds, optimizer and evaluation, so the difference comes from the attribution term rather than data volume or training pipeline. At 10% over-refusal, LAR lowers mixed-decision rows from 25% to 19% (−6.8 points, 95% CI −12.0 to −2.8) and flips from 11.3% to 9.3% (−2.7 points, −5.1 to −0.5); the threshold-free refusal score's relative change under paraphrase falls from 0.306 to 0.206.
The method adds a penalty to the cross-entropy loss that asks the model's attribution vectors for the original and paraphrased request to agree. Attributions are pooled per distinct token rather than per position, because paraphrases change tokenisation and word order, and positional alignment would become a measurement artefact. Training fine-tunes Llama-3.2-1B-Instruct with LoRA (rank 16, α=32) on 300 harmful behaviours from AdvBench and 299 benign instructions from Dolly, each with up to three automatic paraphrases, at regularization strength λ=0.5 for two epochs.
The stability is not a safety gain: the model's ability to tell harmful from harmless requests does not change significantly. At matched over-refusal, harmful refusal rates are statistically unchanged against the control, and detection AUROC rises by 0.019 with an interval crossing zero. AUROC moves from 0.854 to 0.873 (+0.019, 95% CI −0.003 to +0.039), and the gain concentrates on prompts carrying a call-tool-first hint, where flips fall from 15.5% to 8.7%.
Perspective
The findings rest on Llama-3.2-1B-Instruct, LoRA fine-tuning, a single regularization strength λ=0.5 and three seeds, evaluated on AgentHarm first-turn decisions with stub tools and matching thresholds calibrated on AgentHarm's harmless tasks.
The author frames the results as preliminary: intervals reflect behaviour sampling and pooling of three runs, with only three seed pairs; decisions are first-turn only, judged by a keyword rule and tool-call detection without AgentHarm's grader; evaluation paraphrases come from one generator and a human equivalence audit is still needed; the untouched model refuses rarely, so absolute refusal rates here say little about stronger models.