Skip to main content
Back to timeline
researchsquare.comSource publication:

Right tool, swapped argument: BiCP cuts confirmed injected attacks from 83/682 to 0/682 on a 922-case fixed replay

Synopsis

Tool-using language-model agents can pick the intended tool yet execute an injected argument, and existing counterfactual guards lose source sensitivity when they score with a reasoning prefix that already encodes the injected decision. BiCP removes the current-step reasoning prefix before scoring, probes both the tool-name target and the complete-call target, and routes proposals by operation risk. On a 922-case fixed replay, the full method reduces confirmed InjecAgent attacks from 83/682 to 0/682 and unsafe conditional executions from 33/240 to 0/240, while benign task success rises from 66/120 to 78/120.

Source-provided article image: BiCP: Reasoning-Prefix Deconfounding for Dual-Axis Counterfactual Probing and Operation-Risk Routing
researchsquare.com

Interpretation

The pre-execution guard now checks the tool name and the complete call arguments together, so a substituted argument under the same tool can be stopped. A gate that scores only the tool name is structurally blind to same-name argument substitution; the earlier name gate blocked only two of 253 known unauthorized scored proposals and missed all five malicious parameter substitutions the model actually adopted. On the 922 fixed replay cases, the full method reduces confirmed InjecAgent attacks from 83/682 to 0/682 and unsafe conditional executions from 33/240 to 0/240, while benign task success rises from 66/120 to 78/120.

Dropping the current-step generated reasoning prefix before scoring lets a source mask actually change the decision signal being measured. If the same reasoning stays fixed during scoring, masking the external observation leaves a second representation of the injected decision in context, so source ablation can miss the change it was meant to measure. Across 69 fixed historical proposals, an equal-token neutral prefix raises tool-name macro AUC from 0.7164 to 1.0000; Mistral-7B-Instruct-v0.3 raises the same figure from 0.6509 to 1.0000 on the same fixed proposals and reproduces source reversals in ten families across five domains, joint p = 0.03125.

Proposals are routed by operation class: scoped discovery skips model probing, sensitive reads enter the probe, and mutations first require public-evidence anchoring. Previously every proposal went through the same scoring path, so authorized discovery also paid model forward-pass cost. Complete-call prefix slicing cuts scoring forwards from six to three, measured at roughly 1.95 to 1.96 times speedup on T4 GPUs, and the authorized discovery path adds no model forwards.

Perspective

The demonstrated scope is fixed-proposal replay and fixed scoring events under simulated tool execution: the 922-case longitudinal benchmark, 390 independent calibration blocks, a 28-unit pilot, and fixed-target mechanism comparisons on Qwen2.5-7B-Instruct and Mistral-7B-Instruct-v0.3; real external tool execution is zero.

The 922 cases are a longitudinal standard benchmark whose preliminary name-only outcomes had already been inspected, so this is not a fresh blind confirmation; the calibration-fitted 2/75 false rejections and zero observed benchmark attacks are not population guarantees. End-to-end generation and deployment results come from Qwen2.5-7B, with Mistral replicating fixed-proposal mechanisms only; the 47 reserved native tasks, cross-model end-to-end generation and adaptive attacks remain future work.