Attention as the battleground for RAG poisoning: steer it, or read it
A single poisoned passage can hijack a RAG answer by capturing the model's attention. New work turns that same attention into a detection signal — and a way to wall documents off from each other.
What is this?
Retrieval-augmented generation (RAG) grounds a model’s answer in documents pulled from a corpus at query time. Corpus poisoning is the matching attack: an adversary plants one or more crafted passages so that when they are retrieved, they steer the generated answer. The open question has always been why a single passage buried among many legitimate ones can dominate the output — and a converging line of 2025–2026 research gives a concrete answer: attention. A poisoned passage wins when it captures a disproportionate share of the model’s attention during generation. That single mechanistic insight now drives both a more scalable attack and two defenses that read or reshape attention directly. The most recent piece, a sparse-attention defense, was posted in February 2026, and the attention-aware detection line was presented at ICML 2026.
How it works
The attack side was sharpened by Eyes-on-Me (preprint October 2025, revised December 2025). Instead of re-optimising a whole poisoned document for every target phrase — the expensive step that made earlier corpus poisoning scale badly — it decomposes an adversarial document into two reusable parts: Attention Attractors and a Focus Region. The attractors are optimised once to pull a small subset of attention heads — the ones empirically most correlated with attack success — toward the focus region, into which the attacker later drops a retriever bait or a generator instruction. Reported across 18 end-to-end RAG settings, this let a single document in a 1,000-document corpus (≤ 0.1% contamination) redirect answers, with the costly optimisation amortised across many targets.
The defensive insight is the mirror image. If a successful attack must over-concentrate attention, then attention is observable evidence of the attack. Through the Stealth Lens (preprint mid-2025, ICML 2026) formalises this as a distinguishability game and introduces two lightweight tools: a Normalized Passage Attention Score (NPAS) that estimates how much each retrieved passage influences the answer, and an Attention-Variance (AV) Filter that flags passages whose influence is anomalously high relative to the rest. The paper’s core claim is a trade-off: an attack that works despite corrupting only a tiny fraction of the input cannot also be perfectly stealthy, because outsized effect shows up as an outsized attention footprint.
A February 2026 paper attacks the problem structurally rather than by detection. Its Sparse Document Attention (SDAG) replaces standard causal attention with a block-sparse mask that forbids cross-attention between separately retrieved documents — each passage may attend within itself and to the query, but not to its neighbours. The reasoning: poisoning is not only a content problem but an interaction problem inside the generation mechanism, where a malicious document leans on benign ones to build a convincing answer. Cutting those cross-document paths is an inference-time mask change — no fine-tuning, no architecture surgery — and the authors report a substantial drop in attack success rate versus dense attention.
Why it matters
RAG is now the default pattern for grounding assistants in private or fast-moving data, which means the corpus is an attack surface: wikis, ticket systems, scraped web pages, and shared drives are all writable by someone. Framing the fight around attention changes the defender’s options in two useful ways. First, it yields a model-internal signal that does not depend on knowing the attacker’s trigger in advance, unlike keyword or content filters that an adaptive adversary edits around. Second, SDAG shows a structural mitigation that removes a whole class of cross-document manipulation without retraining. The caveat runs both directions: Eyes-on-Me already steers specific heads, so an attacker who models the defender’s attention statistics may try to spread influence and stay under a variance threshold. This is an arms race over the same quantity, not a solved problem — which is why the recent survey work catalogues attention-based methods as one family among many rather than a silver bullet.
Defenses
No single control is sufficient; layer them.
- Score passage influence, don’t just match content. Compute a per-passage attention/influence score (e.g. NPAS-style) at generation time and flag passages whose influence is far above the batch’s variance. Treat outliers as suspect and re-run without them to compare answers.
- Isolate documents in the attention mask. Where you control the serving stack, evaluate sparse/block document attention so retrieved passages cannot cross-attend; it is an inference-time change and composes with detection.
- Assume the corpus is writable. Gate ingestion: provenance and signing for documents, allowlists for sources, and review for anything user- or web-supplied before it enters the index.
- Test with attention-aware attacks. Red-team with transferable, low-contamination attacks (single-document, ≤ 0.1%) and with adaptive variants that deliberately flatten their attention footprint, not just obvious keyword injections.
- Keep humans and diffs in the loop for high-stakes answers. Log which passages were retrieved and their influence scores; alert when one low-ranked document dominates the answer.
- Don’t rely on one signal. Combine attention scoring with retrieval-time filtering, output verification, and provable-aggregation approaches; adaptive attackers optimise against whichever single defense you deploy.
Status
| Item | Reference | Date | Notes |
|---|---|---|---|
| Eyes-on-Me (transferable attention-steering attack) | arXiv 2510.00586 | 2025-10 (rev. 2025-12) | Reusable Attention Attractors + Focus Region; single doc at ≤0.1% corpus |
| Through the Stealth Lens (NPAS + AV Filter) | arXiv 2506.04390 | preprint 2025, ICML 2026 | Distinguishability game; attention as a detection signal |
| Sparse Document Attention (SDAG) | arXiv 2602.04711 | 2026-02 | Block-sparse mask forbids cross-document attention; inference-time only |
| RAG security taxonomy (context) | arXiv 2604.08304 | 2026 | Situates attention-based methods among broader defenses |
The through-line is that RAG poisoning and its most promising defenses now argue over the same variable. Attackers win by concentrating the model’s attention on a planted passage; defenders can watch that concentration as an anomaly, or forbid the cross-document interactions that let a poisoned passage borrow credibility from honest ones. Neither defense closes the game — an adaptive attacker can try to spread its footprint — but reasoning about attention gives operators a signal and a structural lever they did not have when corpus poisoning was treated purely as a content-filtering problem.