Phone agents fall to cross-modal injection: same payload in screenshot and UI text hits 85.4% attack success, single-channel repeats lag
Synopsis
For multimodal-LLM phone agents, MobileInject places one malicious objective in both the screenshot and the interface text and runs 108 executable risk instances on a resettable Android emulator. Prior evaluations mostly watched a single text channel or judged only whether an attack succeeded; with exposure counts matched, one appearance per channel reached 85.4% attack success and 0% benign completion, while repeating within one channel reached only 72.9% or 27.8%. In 30 of 108 paired instances, neither single channel succeeded but the aligned pair did.
Interpretation
Security evaluation of phone agents moves from single-channel text injection to the case where visual and textual observations are poisoned together, inside an executable, resettable Android setting with verifiable endpoints. Earlier agent-injection evaluations centered on desktop and web settings, or covered only one textual channel and one injection carrier, so they could not capture the risk when a phone agent reads both the rendered screen and the structured interface description. The environment spans 11 application scenarios, 9 security-risk categories and 108 executable risk instances, each with a resettable snapshot and two deterministic oracles that separately judge benign and adversarial completion.
When the same malicious instruction appears consistently in the screenshot and the interface text, the attack works better than either channel alone. Single-channel injection puts the malicious content only on the visual or the textual side, whereas a phone agent during normal execution consumes both the rendered and the structured view of the same interface state. With exposure counts matched, repeating the text twice raised attack success from 68.8% to 72.9%, repeating the visual payload stayed at 27.8%, and one appearance in each channel reached 85.4% with 0% benign completion; 30 of 108 paired instances reversed from failure under both single channels to success under the aligned pair.
The attack rewrites the rationale the agent writes out before it propagates into executed actions, so diagnosis moves from endpoints to step-by-step trajectories. Earlier evaluations mostly used coarse binary judgments that answered only whether an attack succeeded, leaving unclear whether the agent noticed the risk, when its reasoning began to follow the adversarial objective, and whether that drift entered executed actions. Across nine multimodal models paired with two agent frameworks, average reasoning hijack rate was 75.31% under MobAgent and 80.86% under M3A, and average action hijack rate was 70.78% and 67.59%; GLM-4.5V showed identical reasoning and action hijack rates in both frameworks, 98.15% and 99.07%.
Perspective
Evaluation runs in a resettable Android emulator, with instances built around nine benign workflows: messaging, contacts, email, social posting, notes, file management, calendar, paper trading and message-to-web navigation, and adversarial objectives limited to 12 registered goals across 9 consequence categories. The intended readers are teams studying phone-agent security and multimodal injection, and engineers who need to assess the risk boundary of an agent framework.
The 108 instances reuse workflow and objective structures, so they are not semantically or statistically independent, and the analysis accounts for this. Prompt-level defenses offer limited protection: SCoT cut attack success by 12.96 points, but its cluster interval crosses zero and its Holm-adjusted value is non-significant, and defended runs can still fail the benign task. Reported repeated-run variance is limited to the displayed metrics and sampled reruns.