When a poisoned agent skill hides in the false alarms

What is this?

Agent “skills” are a lightweight way to extend a general-purpose agent: a folder with a short description, some prose instructions, and occasionally a bit of code, distributed through open marketplaces and repositories. That openness is also an attack surface. A poisoned skill can carry an attacker’s instruction that the agent executes while doing what the user actually asked.

A paper published on June 6, 2026 by a team at the University of Illinois Chicago — POISE: Position-Aware Undetectable Skill Injection on LLM Agents (arXiv:2606.07943) — studies why some of these poisoning attempts succeed and others get caught, and lands on an uncomfortable conclusion: the static LLM scanners many teams rely on to vet skills can be evaded not by hiding harder, but by blending into the noise those scanners already produce.

How it works

The researchers measure attacks with a strict definition of success: the injected payload has to actually run and the user’s legitimate task still has to pass its verifier in the same trial. That second condition matters. If executing the payload derails the user’s task, the visible failure invites someone to open the skill and look — so a “successful” injection is one that stays invisible.

Under that lens, prior skill-poisoning techniques hit a reliability-versus-stealth trade-off. Injections placed in a skill’s YAML header load reliably but are trivial to inspect. Injections buried in the skill’s body prose are stealthier, but tend to be unreliable: an out-of-context command sitting in the middle of otherwise sensible instructions can trip the agent’s own suspicion and get ignored.

The paper’s contribution is positional rather than a new payload. Instead of dropping an obvious command into the prose, the technique compresses the trigger into a single, benign-looking body instruction and places it where such an instruction would plausibly belong — next to setup or prerequisite steps — using a context-aware generator so it reads like a natural part of the skill. We are deliberately not reproducing the construction here; the lesson is the placement principle, not a copy-paste recipe.

The reported numbers are what make it notable. On the authors’ Skill-Inject benchmark, the approach reached an 89.3% attack success rate, well above a random-placement body baseline and marginally above a header-only baseline, while keeping the stealth of body placement. The decisive finding is about the defenders: because legitimate skill bodies routinely ask for privileged tool operations, the LLM scanners meant to catch abuse are hyper-sensitive and falsely flagged 74.6% of clean skills on average across four judge models and two benchmarks. Poisoned variants blend into that flood of false positives — only 5.6% of them raised a new high-risk alert over their clean baseline. A scanner that alarms on everything effectively alarms on nothing.

Why it matters

This is a supply-chain problem dressed up as a content-moderation problem. Teams are increasingly gating third-party skills through an “LLM security scanner” step and treating a clean verdict as permission to run. The research suggests that verdict is close to meaningless when the base rate of false alarms is that high: operators either drown in noise or quietly raise the threshold until real attacks pass through with everything else.

It also generalizes past this one benchmark. Any pipeline that relies on a static, pre-execution language-model review of untrusted instructions inherits the same weakness — the reviewer cannot cleanly separate “a skill that legitimately needs to touch the filesystem and network” from “a skill that abuses exactly those capabilities,” because on the page they look alike. The distinguishing signal is in what the skill does at runtime, not in how its prose reads beforehand.

Defenses

Stop treating a scanner’s “clean” as authorization. A pre-execution LLM review is a weak filter, not a gate. If your only control is a static scan, the research indicates you are exposed. Measure your scanner’s false-positive rate on known-good skills before you trust its negatives — a detector that flags most clean inputs carries almost no information when it stays quiet.

Move the boundary to runtime authority. Enforce least privilege on what a skill can actually do: scope filesystem and network access per skill, require explicit capability grants rather than ambient permissions, and put human confirmation in front of privileged or irreversible tool calls. Complementary research this quarter — for example internal-signal detectors that watch the agent’s own activations at execution time (RouteGuard, arXiv:2604.22888) — points the same direction: catch the behavior, not the wording.

Establish provenance and integrity. Prefer signed skills from known publishers, pin versions, and diff updates so a quietly edited body instruction is visible in review. Treat an unsigned or freshly-published skill from an anonymous source as untrusted by default.

Sandbox execution. Run skills in an isolated context with dropped capabilities, no standing credentials, and no default outbound network, so that even a payload that executes cannot reach secrets, metadata endpoints, or exfiltration paths.

Log and audit behavior, not just intent. Record the tool calls a skill makes and compare them against what the skill claimed to need. Post-hoc behavioral auditing catches the cases a pre-flight text review structurally cannot.

Status

Item	Detail
Type	Academic research (skill-poisoning attack + defense analysis)
Published	June 6, 2026 (submitted June 11, 2026)
Source	arXiv:2606.07943 — University of Illinois Chicago
Tested on	Skill-Inject benchmark, codex + gpt-5.2
Key result	89.3% attack success rate; 74.6% false-positive rate on clean skills across scanners; 5.6% of poisoned skills raised a new high-risk alert
Code	github.com/liofoil/SkillSafety
Takeaway	Static LLM scanners are unreliable skill gates; enforce runtime least-privilege and behavioral auditing