Browser agents now resist hand-crafted injection — coding agents don't

What is this?

A preprint posted to arXiv in June 2026 by Nicholas Saban (Patronus AI / UC Berkeley), Domain-Conditioned Safety in Frontier Computer-Using Agents, asks a question every practitioner deploying an agent should be asking: when a red-teaming paper reports a 90%-plus prompt-injection success rate, does that number still hold on the model you are actually running today?

The short answer the paper documents is no — and the reason is uncomfortable for the field. The headline attack success rates in the computer-use-agent (CUA) literature, which range from roughly 42% to 98%, cluster on retired models and on whichever model was weakest in each paper’s panel. Reproduced as plain hand-written templates against current frontier models, most of those techniques stop working entirely. But that hardening turns out to be narrow: it lives on the browser surface the vendors have been stress-testing, and it does not carry over to coding-agent skill files.

How it works

The study releases CUA-HANDCRAFTED, a public benchmark of 793 episodes spanning 24 multi-step web tasks across 8 sites, 56 attack templates drawn from 8 attack families, 5 injection-depth levels, and 4 system-prompt configurations. Crucially, the attacks are not new: they are human-readable reconstructions of the techniques described in prior red-teaming papers (RL-Hammer, WASP, TRAP, RedTeamCUA, MUZZLE, pop-up overlays), transcribed from those papers’ prose rather than replayed from released strings.

Run against two current frontier models, the browser result is a floor:

Benchmark surface        Model            Hand-crafted ASR
-----------------------  ---------------  --------------------------------
Browser (multi-step)     Sonnet 4.6       0 / 140  (95% upper bound 2.60%)
Browser (multi-step)     GPT-5.4          included in the same 0/140
Browser, prompt ablation L0_bare/L1_help  still 0% — resistance is weight-level
Coding skills (SKILLBENCH) Sonnet 4.6     up to 40 / 40  = 100%
Coding skills (SKILLBENCH) GPT-5.4        up to 79 / 100 = 79%

Two design choices make the browser result credible. First, a prompt ablation: stripping the system prompt down to a bare or merely-helpful configuration does not reopen the attacks, which means the injection-resistance is baked into the model weights, not bolted on by a defensive system prompt. Second, an RL-attacker ceiling: an in-harness adaptive-random-suffix attacker (an AutoInject-style baseline) also scored 0/100 within a five-query-per-target budget and about $10 of API spend — evidence that the gap between hand-crafted and published attacks is the optimized phrasing, not the attack category. A 50-episode replication of the popup-overlay image-channel class (VPI-Bench) landed at roughly 3% on the frontier, closing an image-channel attack that older models fell for. No optimized payloads are reproduced here; the interest is entirely in the measurement.

Why it matters

The paper reframes a number that security teams have been quoting to each other. The literature’s 42–98% attack success rates appear to be largely a function of RL-discovered, off-distribution injection strings, not of the attack ideas those papers name. When the string is not released — and the audit finds that “retired target model plus unreleased strings” applies to four of the six papers examined — the headline number is not reproducible on the model you deploy. That is a sourcing problem as much as a security one, and it is the kind of claim a defender needs to be able to check before budgeting around it.

The more consequential finding is domain-conditioned safety. The same Claude Sonnet 4.6 weights that resist browser injection at 0/140 fall to hand-crafted skill-injection in a coding-agent harness at up to 100%; GPT-5.4 falls at up to 79%. Anthropic’s own Sonnet 4.6 system card reports browser injection dropping from about 49% to roughly 1.3% between two model generations — a real 38× improvement, but one that this work shows is specific to the browser modality. Vendors have been hardening the surface researchers hammer hardest, and that effort does not automatically generalize to the tool-call and skill-file surfaces that coding agents expose. It is the tool-vs-tool extension of the earlier observation that text-level safety does not transfer to tool-call surfaces.

The practical reading: do not extrapolate a browser-agent safety score to your coding agent, your MCP tools, or your agent skill pipeline. They are different threat surfaces with different — and, on today’s evidence, much weaker — hardening.

Defenses

• Treat published ASR numbers as surface-specific. A low browser-injection rate on a frontier model tells you little about that same model inside a coding-agent or MCP harness. Re-measure on the surface you actually deploy, and prefer benchmarks that publish their attack strings so you can reproduce them.
• Harden coding-agent skill ingestion independently. Since skill-injection is where the frontier weights still fail, apply provenance checks, review or sandboxing of third-party skills, and least-privilege tool scoping there — do not assume model-level alignment covers it. See our coverage of malicious agent skills and skill red-teaming.
• Keep out-of-band controls in place regardless of model score. Model-weight resistance is welcome but brittle across domains; pair it with adaptively-evaluated out-of-band defenses, egress limits, and human-in-the-loop confirmation for consequential actions, consistent with the lethal-trifecta framing.
• When you red-team, release the strings. Reproducibility is a defense: an attack you cannot re-run is one you cannot regression-test after the next model update. Report the optimized text, the exact model version, and the harness, or the number decays into folklore.
• Watch for benchmark disagreement. This result sits alongside a broader finding that safety benchmarks disagree with one another; triangulate across several before trusting any single headline.

Status

Item	Detail
Publication	arXiv preprint, June 2026 (Saban, Patronus AI / UC Berkeley)
Browser ASR (frontier)	0/140 multi-step; 95% Clopper–Pearson upper bound 2.60%
Coding-skill ASR (same weights)	up to 100% (Sonnet 4.6), up to 79% (GPT-5.4)
Benchmark released	CUA-HANDCRAFTED — 793 episodes, 56 templates, 8 attack families
Reproducibility audit	”retired target + unreleased strings” in 4 of 6 audited papers
Vendor reference	Anthropic Sonnet 4.6 system card: ~49% → ~1.3% browser injection ASR

This is a research/benchmark finding, not a product vulnerability; no CVE applies. Model versions and figures are as reported by the cited preprint and vendor system card.