Polymorphic Sybil poisoning slips past RAG duplicate filters
A July 2026 benchmark shows coordinated, lexically varied passages defeat the near-duplicate filters that catch simple RAG poisoning — and that attack-success scores hide most of the damage.
What is this?
In July 2026, researchers posted to arXiv a benchmark and evaluation framework for grounded question answering under coordinated retrieval poisoning — attacks that plant hostile documents in the corpus a retrieval-augmented generation (RAG) system reads from. Their headline contribution is a named attack class, polymorphic Sybil poisoning, and a measurement result: the metrics most teams use to score RAG robustness systematically understate how badly a poisoned system behaves. The work builds on a growing 2026 literature on RAG corpus poisoning, including earlier benchmarking efforts and corpus-dependent attacks, but it is the first to isolate what surface-level diversity among colluding passages does to both attacks and defenses.
How it works
Classic RAG poisoning plants one hostile passage — or several near-identical copies — that a retriever surfaces for a target question, nudging the reader model toward an attacker-chosen answer. A common, cheap defense is lexical near-duplicate filtering: if many retrieved passages are textually almost the same, drop the duplicates. That filter fully catches the monomorphic case, where the attacker reuses one string.
A Sybil attack sidesteps it. Instead of one passage repeated, the attacker seeds many lexically diverse passages that all argue for the same target claim. Each looks like an independent source; together they form a fake consensus. Because no two are near-duplicates, the deduplication filter that stops the copy-paste version sees nothing to remove. The researchers call this surface variation polymorphism and treat monomorphic-versus-polymorphic as the key axis.
To measure the effect cleanly, they separate two things that normally blur together: whether the poison gets retrieved, and how the reader resolves the conflict once poison is in front of it. A “Forced Exposure” protocol fixes the retrieved context so evaluation isolates reader-side behavior from retrieval noise. Every output is then sorted into four mutually exclusive buckets — gold (correct answer), hijack (the attacker’s answer), abstention (the model declines), and drift (some other wrong answer) — with paired clean-to-poison transition matrices tracking how each question moves when poison is introduced.
Why it matters
Two results stand out. First, polymorphism pays. In a controlled ablation, monomorphic copies registered as a successful hijack only 4.0% of the time, while giving the same malicious claim diverse surface forms recovered 22.8% — a +18.8 percentage-point jump, roughly a 5.7× amplification of the attack channel that a standard attack-success-rate (ASR) metric can see. The defense that neutralizes the naive attack barely dents the coordinated one.
Second, ASR is the wrong ruler on its own. Under attack, the “abstention” and “drift” buckets together held between 47% and 66% of all output mass — failures that never show up as a hijack. Two reader models sitting at nearly identical ASR differed by 16.5 points on abstention and 17.2 points on drift. In other words, two systems that look equally robust by the headline number can fail in completely different ways, and a team optimizing only for low ASR is blind to most of what poison actually does to their pipeline. To make this reproducible, the authors released a frozen benchmark of 3,145 questions and 2,982 retained Sybil groups spanning five reader models from 7B to 120B parameters, two retrievers, and two cross-validation setups.
Defenses
The practical lesson is that near-duplicate filtering and ASR-only evaluation give false confidence against coordinated poisoning. Treat lexical deduplication as a floor, not a defense: add semantic-level clustering that can spot many differently-worded passages pushing one claim, and weight retrieved evidence by source provenance and independence rather than by count, so a swarm of fabricated “sources” cannot manufacture consensus. On the reader side, prefer conflict-aware prompting that surfaces disagreement between sources and permits abstention when support is thin or contradictory, and require citation grounding so an answer must point to identifiable evidence.
On evaluation, stop scoring robustness with a single number. Track abstention and drift alongside hijack, use paired clean-to-poison transitions to see how individual questions move, and red-team retrieval pipelines with polymorphic variants — not just repeated strings — before deployment. The released benchmark gives defenders a ready way to do exactly that. Finally, harden the ingestion path: monitor what enters the corpus, apply trust tiers to sources, and cap how much any single unverified origin can contribute to the evidence behind one answer. These controls map onto OWASP’s guidance on data and knowledge-base poisoning for LLM applications.
Status
| Item | Detail |
|---|---|
| Contribution | Failure-mode benchmark + evaluation framework for RAG poisoning |
| Attack class | Polymorphic Sybil poisoning (coordinated, lexically diverse passages) |
| Output taxonomy | Gold · hijack · abstention · drift |
| Key result | Polymorphism amplified visible hijack ~5.7× (4.0% → 22.8%) |
| Hidden failures | Abstention + drift held 47–66% of outputs under attack |
| Benchmark | 3,145 questions · 2,982 Sybil groups · five readers (7B–120B) · two retrievers |
Key date: benchmark paper posted to arXiv in July 2026.