TRACE: catching RAG corpus poisoning by following token influence

What is this?

TRACE is a lightweight detection method for corpus-poisoning attacks against Retrieval-Augmented Generation (RAG) systems. It was introduced in “Tracing Target Answers in Poisoned Retrieval Corpora via Token Influence Attribution” (arXiv:2606.25721, cs.CR), a paper published in late June 2026. The idea is defensive and forensic: instead of trying to catch a poisoned document by how it reads, TRACE catches it by how much it influenced the answer the model produced.

Corpus poisoning is the family of attacks where an adversary slips crafted documents into the knowledge base a RAG system retrieves from, so that when a victim asks a targeted question, the poisoned passage gets retrieved and steers the generated answer toward an attacker-chosen output. It is one of the more durable RAG threats because the malicious text can look perfectly ordinary to a human reviewer and to perplexity filters. TRACE joins a small but growing line of “attribution” defenses — alongside AttnTrace and earlier traceback work such as RAGForensics — that treat detection as a question of provenance rather than content.

How it works

The core observation is that a poisoned document, to succeed, must exert an outsized pull on the tokens the model emits. TRACE makes that pull measurable. At a high level it runs in two stages, entirely at inference time and without touching model weights.

First, it performs token influence attribution: for a given answer, it estimates how strongly each retrieved token contributed to the answer tokens, then looks for recurrent high-influence keywords that show up across the retrieved set with anomalously large influence. In a clean corpus, influence tends to be diffuse; a targeted poison concentrates it on the small span of text carrying the attacker’s payload.

Second, it runs a verification pass that confirms whether those flagged high-influence tokens actually drive the model’s prediction, rather than being coincidental. Only tokens that survive this second check are treated as evidence of poisoning.

Two properties make this attractive operationally. It does not require an auxiliary classifier or a second LLM acting as a judge, so it avoids the compute overhead and added attack surface those approaches carry. And because the method centers on the tokens that produce the answer, it surfaces the attacker-specified target answer as a by-product — you learn not just that the corpus was poisoned, but what the poison was trying to make the system say. The authors report strong detection across three question-answering benchmarks and six different LLMs.

Why it matters

RAG is now the default way enterprises wire private data into an LLM, which makes the retrieval corpus a first-class attack surface. Anyone who can write to an indexed source — a wiki, a ticketing system, a shared drive, a crawled web page — is a potential poisoner, and the payoff is high: quiet, targeted manipulation of answers for specific queries while everything else behaves normally.

Most deployed defenses sit at the retrieval step (filtering or re-ranking documents before generation) or lean on a separate model to screen inputs. Those help, but they add latency and cost, and they miss poisons engineered to look benign until generation. A detection method that works at the answer step, is cheap enough to run inline, and doubles as an incident-response tool — telling responders which document to pull and what the attacker wanted — fills a real gap. The same influence signal that flags an attack also gives a defender the artifact needed to purge the offending record and audit the blast radius.

Defenses

TRACE is itself a defensive contribution, but the practical takeaway is to layer attribution into a RAG pipeline rather than rely on any single control.

• Attribute answers to sources. Track, per answer, which retrieved documents and which spans drove the output. This is the capability TRACE, AttnTrace and RAGForensics-style traceback provide, and it is what lets you locate and remove a poisoned record after the fact.
• Keep filtering at retrieval, but don’t stop there. Chunk-level anomaly and perplexity filtering still catches clumsy poisons before generation; pair it with answer-step attribution to catch the ones designed to pass content inspection.
• Score source provenance. Weight documents by the trustworthiness of where they came from, and treat writable, low-trust sources (public wikis, user-submitted content, crawled pages) as higher-risk during retrieval.
• Instrument for response, not just prevention. Log influence attributions so that when a suspicious answer surfaces, an analyst can trace it back, identify the target answer, and clean the corpus — turning detection into remediation.
• Re-test defenses under a realistic pipeline. Evaluate any RAG defense end-to-end (chunking, retrieval, reranking, generation), since attacks and defenses both behave differently once the full pipeline is in play.

Status

Item	Reference	Date	Note
TRACE (this work)	arXiv:2606.25721	2026-06	Token-influence detection + target-answer recovery; 3 QA benchmarks, 6 LLMs
AttnTrace	arXiv:2508.03793	2025-08	Attention-based contextual attribution of injection/corruption
Needle-in-RAG	arXiv:2605.01782	2026-05	Character-level traceback of poisoned spans
RAGForensics / traceback	arXiv:2504.21668	2025-04	Attributes answers back to retrieved documents