When the pentest bites back: attacking the tools that red-team for you

What is this?

A preprint published on 23 June 2026 (arXiv:2606.24496) presents what its authors call the first in-depth security analysis of the autonomous agents now used to run offensive-security operations. The community has spent two years making these “agentic red-team” tools more capable; far less attention has gone to whether the tools themselves are safe to run. The finding is uncomfortable: most of the widely used ones share the same structural flaws, and a defender sitting on the target of a penetration test can turn the attacker’s own automation against them.

The threat model is the ordinary one for a pentest. The operator points an autonomous agent at a target and lets it work. What the study adds is the observation that in a black-box engagement the target is untrusted input the agent must interact with — and interacting with untrusted input is exactly where agents get compromised. The result is a role reversal: the machine running the offensive tooling, not the target, is the one that ends up owned.

How it works

Agentic red-team platforms tend to share a three-part shape: an orchestrator that runs the agent loop and holds the sensitive assets (LLM API keys, cross-session logs, skills, memory), one or more worker nodes — usually Kali containers — that actually execute shell commands and offensive tools against targets, and an operator front-end. The workers touch the outside world; the orchestrator holds the crown jewels.

The manipulation technique the authors describe, which they call “agent-phishing,” deliberately avoids classic prompt injection. Instead, a defender stages plausible artifacts on a honeypot — a password-vault helper, a database-restore utility, a config unpacker — so they look like exactly the kind of thing the agent needs to make progress. The agent discovers them, downloads them to its own infrastructure, inspects them, and runs them, because from its point of view that is how the engagement advances. The staged tool does what it claims (it decrypts a password and exits cleanly), but parsing a crafted input file triggers a memory-safety defect that hands control to the attacker as a silent side effect. Because the binary carries an engineered implementation bug rather than obvious malware, both static and dynamic analysis tend to wave it through. This detail matters, so we will keep it abstract: no working payload is reproduced here.

Once code runs on a worker, the paper lays out an escalation chain: from worker to orchestrator through incomplete filesystem isolation (shared writable volumes exposing orchestrator code) or weak network isolation (host networking, reachable internal APIs); persistence, either by trojanizing source and config that survive container resets or by poisoning long-lived skills and episodic memory so future sessions reinfect themselves; sandbox escape via over-permissive container settings such as --privileged, CAP_NET_RAW, or a mounted Docker socket; and finally full host compromise.

Why it matters

Across an automated testbed spanning ten of these tools and six frontier models — including Claude Opus 4.8, GPT-5.5, and Gemini 3.1 Pro — staged payloads reached code execution inside the worker in 97.8% of completed runs. The only reliable failures came when the model refused to perform the penetration test at all; once a test genuinely began, the agent almost always found, fetched, inspected, and ran the planted tool. The blast radius is real because of what these systems hold: an offensive agent’s orchestrator is a concentration of API keys, client logs, and reusable attack skills, and its workers already have permission to generate hostile network traffic. A compromise turns a sanctioned red-team platform into attacker-controlled infrastructure.

Worse, the guardrails many of these products advertise do not help after the worker falls. Policies are enforced at the orchestrator layer — for example, blocking scans against .gov domains — while the worker’s actual OS-level behavior goes unmonitored. An adversary with a shell on the worker simply acts beneath the guardrail’s line of sight.

Defenses

The authors’ guidance is to stop treating the model-controlled worker as trusted and to engineer for a small blast radius when — not if — it is compromised:

Assume the worker is hostile. The component that ingests untrusted target data should be the least privileged part of the system. Design as though an attacker already has a shell on it.

Keep secrets off the workers. LLM API keys, client logs, and credentials belong on the orchestrator or in a secrets service, never mounted into the container that touches targets.

Enforce isolation at the OS and network layer, not just in prompts. Route worker egress through an external proxy that applies policy on real traffic, drop over-permissive container capabilities (--privileged, extra CAP_*, mounted Docker sockets), and give workers scoped, minimal filesystem and network access.

Don’t run fetched tools on the orchestrator. Avoid executing target-supplied binaries anywhere near the trusted core; expose high-capability actions only through narrow, hardened APIs rather than a raw shell.

Treat memory and skills as an attack surface. Persistence through poisoned episodic memory or skills is a documented path — sign, scope, and review long-lived agent inputs so a single session cannot silently reinfect the next.

Status

Item	Reference	Notes
Core study	arXiv:2606.24496, 23 Jun 2026	First in-depth security analysis of widely used agentic offensive-security tools
Technique	”Agent-phishing”	Contextual deception, not prompt injection; agent runs a staged tool itself
Measured result	Same paper	Worker code execution in 97.8% of completed runs across 10 tools, 6 models
Evaluated models	Same paper	Includes Claude Opus 4.8, GPT-5.5, Gemini 3.1 Pro
Independent coverage	GBHackers, 25 Jun 2026; Adversa AI digest, 2 Jul 2026	Public reporting of the disclosure

The durable lesson generalizes past offensive tooling: any agent that inspects and executes artifacts from an untrusted environment is one crafted file away from running attacker code. The safe assumption is that the part of your system touching the outside world will be compromised — build so that when it is, the damage stops there.