Role Spoofing: When Tool Output Impersonates the User

What is this?

On October 29, 2026, Ezra Edelman, Benjamin Edelman, Surbhi Goel and Eran Malach posted “Understanding and Mitigating Role Spoofing in LLMs” (arXiv, cs.CR). The paper studies a form of prompt injection in which malicious text inside a tool output, such as a web page or an email an agent reads, impersonates the user rather than merely giving orders.

It builds on “role confusion”, the finding (ICML 2026, arXiv:2603.12277) that models infer who wrote a piece of text from its style instead of its labeled source. We covered that earlier work in a previous article; this one adds the structural half of the story.

This is academic research on a design weakness, not a flaw disclosed in a single product.

How it works

The authors find that models decide “who said what” using two kinds of cues: content cues (wording, tone, semantics) and structure cues (formatting and chat-template-like markers). Different models lean on different mixtures.

Role spoofing combines both: text that reads like the user and is dressed in fake structure that imitates the conversation format. Real control tokens cannot appear inside content, so the imitation is ordinary text the tokenizer treats as regular tokens, yet it can still shift the model’s belief about the speaker. The paper does not need, and this article does not reproduce, any working payload.

Reported results, as stated by the authors:

• On a custom tool-use testbed, injection success was 100% for GPT-5.4, Claude Opus 4.8 and Claude Opus 5, 98.8% for Claude Sonnet 5, 98.5% for Gemini 3.5 Flash and 72.5% for Gemini 3.1 Pro.
• On a modified AgentDojo workspace suite, injection-to-control ratios ranged from 63.1% to 103.6%. The authors note the harness was made more permissive, so figures are not comparable to default settings.
• In a toy setting, adversarial training against content-based injections fixed that weakness but left structure-based spoofing working, and fine-tuning Qwen3.5-27B against one injection type transferred little to the other.

Why it matters

Frontier models resist many classic injections, so a defense that works against “ignore previous instructions” style text can create false confidence. This result suggests that training against one family of cues leaves the other open. Because injected text claims the user’s authority, agents may grant it permissions they would deny to a tool result. Any agent that reads untrusted pages, emails or documents in the same context as user requests is in scope.

Defenses

The authors propose detection through linear probes on model activations, trained on synthetic conversations without any injections to predict each segment’s true role. A segment the model internally attributes to a conflicting role is flagged. They report near-perfect separation on held-out tests and better true-positive rates at 0 to 1% false-positive rates than the baselines they compared.

Practical measures for builders:

• Do not rely on one training-time fix. Test both content-style and structure-style spoofing.
• Sanitize and delimit untrusted content. Strip or escape role-marker lookalikes in tool outputs and mark provenance outside the text channel.
• Gate sensitive actions on out-of-band confirmation. Require explicit user approval for high-impact tool calls rather than trusting the apparent speaker.
• Apply least privilege. Limit what an agent can do after reading untrusted content.
• Evaluate probes before trusting them. The paper reports they detect only role spoofing, not other injection types, and are sensitive to query wording.

Status

Item	Detail
Source	arXiv:2610.40396 v1, submitted October 29, 2026
Nature	Academic study across several commercial and open-weight models
Vendor patch	Not applicable; design-level weakness. Detection by probes is a research proposal
Limitations	Probes cover role spoofing only; results use a permissive AgentDojo variant