RIFT-Bench: red-teaming agents by mapping their code, not their prompts

What is this?

RIFT-Bench is a methodology for red-teaming agentic AI systems, published on arXiv on 22 June 2026 by a team at Fujitsu Research of Europe and Fujitsu Research of India. Its argument is that most security testing for LLM agents inherited the wrong assumptions from LLM red-teaming: it treats the user prompt as the only attack surface and judges success only by the model’s final text output. That view fits a chatbot. It does not fit a system that calls tools, keeps memory, coordinates multiple agents, and takes real actions in an environment.

The paper’s core observation is that agent vulnerabilities emerge from the interactions between untrusted inputs, tools, memory, and privileged actions — not from any single prompt. So to test an agent meaningfully, you first have to understand how it is actually built. RIFT-Bench makes that structural understanding the starting point of the evaluation rather than an afterthought.

How it works

The framework runs in two automated phases. The first, Discovery, reads the target system’s codebase and extracts a structured representation the authors call NodeSpec: a hierarchical graph that captures each agentic component, how the components interact, and which blocks of implementation code ground each node. Discovery also sets up configurable tool emulation, so tests can run without actually hitting live external services — a safety and cost measure that keeps evaluation from causing real-world side effects.

The second phase, Scanning, uses the NodeSpec graph to instantiate attacks that fit the specific system, then executes, traces, and scores them, producing a vulnerability report. What makes this reusable across very different agents is a four-level attack-surface hierarchy. A surface is an entry-point class such as tools, memory, external resources, or user input. A suite is an operational configuration within a surface, like injection via tool outputs. A probe is a reusable, system-agnostic template paired with its evaluators. An attempt is a concrete instantiation of a probe against one system’s actual tools and arguments. The same probe can therefore be re-instantiated automatically against many architectures, and paired with a second axis of adversarial objectives: performing an unwanted action, disrupting task completion, leaking internal information, or overloading resources.

Because RIFT-Bench evaluates the system itself rather than a simulated stand-in, its probes reach failure modes that prompt-only testing misses — for example a malicious tool surface expressed as description-level injection, unauthorized actions, or output manipulation, each a distinct suite with its own templates.

Why it matters

The recurring problem in agent security is that evaluations are tied to one framework or one benchmark environment, so results do not transfer. An attack set written for one implementation cannot be reused against another, and comparisons across heterogeneous systems become impossible. RIFT-Bench’s contribution is generality: the authors demonstrate the pipeline across 45 agentic systems spanning five domains — finance, medical, personal-assistant, travel, and open-ended “wild” tasks — and report attack-success rates broken down by attack surface, architecture, framework, attacker objective, and evaluator type. Success rates varied widely across domains and objectives, which is precisely the point: a single headline number hides where a given system is actually weak.

For defenders, the practical value is a repeatable way to ask “where is this agent exposed?” rather than “does this agent fail a fixed test suite?” As agents move from prototypes into enterprise workflows, that shift from prompt-centric to structure-centric evaluation is what lets security testing keep pace with how differently each system is wired.

Defenses

RIFT-Bench is itself defensive tooling, and the paper builds defense evaluation into the framework rather than treating it as separate.

Test defenses in place, not in the abstract. Defenders can define custom intervention logic that RIFT-Bench automatically weaves into the target system, so a mitigation is measured against the same probes as the undefended system. The paper ships a simple description-removal defense (stripping tool descriptions before execution) and integrates prompt sandwiching and data delimiters from prior work as baselines.

Defend at two levels. The authors frame mitigations as either component-level (input/output filtering embedded in individual modules) or system-level (changing the orchestration logic between agents and tools so injections cannot reach privileged actions). Robust agents generally need both; a good classifier on one module does not help if the system design still routes untrusted tool output into a high-authority action.

Map your own attack surface first. The deeper lesson generalizes past this one tool: you cannot secure what you have not modeled. Enumerating where untrusted input enters — user prompts, tool outputs, memory, external resources — and where privileged actions live is the prerequisite to any meaningful test.

Weigh the trade-offs explicitly. Because the framework measures robustness, computational overhead, and task utility together, it makes visible the cost of each defense. Treat mitigation as an engineering trade-off to be measured, not a checkbox.

Status

Item	Reference	Notes
Core paper	arXiv:2606.23927, 22 Jun 2026	Fujitsu Research (Europe and India)
Method	NodeSpec + Discovery/Scanning	Graph representation extracted from the target’s code
Scope	Same paper	Demonstrated across 45 agentic systems, five domains
Defenses evaluated	Same paper	Description removal, prompt sandwiching, data delimiters as baselines
Independent coverage	Adversa AI digest, 2 Jul 2026	Public listing of the work

The durable takeaway is a reframing: an agent’s security is a property of its architecture, not just its prompt filter. Testing that starts by mapping how the system is built will find the interaction-level weaknesses that prompt-only red-teaming structurally cannot see.