The Misattribution Gap: memory poisoning that gets blamed on the model

What is this?

The Misattribution Gap is a structural weakness in how teams diagnose misbehaving AI agents, described in a paper published on 12 May 2026 by researchers at the University of Texas at El Paso, Southern Illinois University Carbondale and the University of Illinois Urbana-Champaign (arXiv:2605.22842, “The Misattribution Gap: When Memory Poisoning Looks Like Model Failure in Agentic AI Systems”). The core observation is uncomfortable: when an agent produces a policy violation because its shared memory was poisoned, the resulting behavior is indistinguishable from a model that has simply become misaligned. Standard AI governance responds the only way it knows how — red-team the model, inspect its attention, retrain — and the violation stops, then returns one reporting cycle later, because the poisoned document is still sitting in the vector store. The paper reports that across all 64 documented failures in its corpus, the attribution machinery confidently blamed the model. The attack that caused the harm was never on the model at all.

How it works

The authors formalize the underlying attack as Semantic Norm Drift (SND), which they argue is a third, structurally distinct path to agent misconduct — orthogonal to emergent misalignment and to covert multi-agent collusion. The mechanism is deliberately mundane. An attacker with nothing more than ordinary document-upload access places a file formatted to look like a legitimate internal policy (their running example is a document dressed up as a SOX §302 compliance policy) into a shared vector store. Through what the paper calls the Trust Laundering Chain, that document is retrieved in later sessions and re-presented to agents as trusted system context, with its original provenance permanently lost along the way. The agents then cite the injected text as normative authority in their own chain-of-thought and comply with it — in the study, 59 of 65 valid entries showed exactly this reasoning-then-comply pattern. There is no trigger phrase, no access to model weights, and no need for repeated interaction: the paper reports full effect within five sessions, sustained indefinitely. We are describing the shape of the attack, not publishing a template; the point is architectural, and no working payload is needed to understand it.

Why it matters

What makes this more than a restatement of known memory-poisoning work is the detection result. The researchers ran four safety classifiers — including one specifically trained to catch memory poisoning — across 510 checkpoints and recorded zero detections. Because the poisoned output is byte-for-byte the kind of thing a genuinely misaligned model might say, output-side classifiers have nothing anomalous to latch onto. The consequence for defenders is a governance trap: retraining after an SND incident consumes real resources, produces a paper trail that looks like successful remediation, and leaves the attack fully in place. In regulated settings the harm is concrete — the paper’s worked example is a three-agent financial-reporting pipeline that begins leaking raw customer identifiers into board summaries, a direct breach of internal data-governance policy and GDPR Article 5. Any enterprise running RAG agents over a shared, writable knowledge store where multiple users or tenants can contribute documents is exposed to the same misdiagnosis loop.

Defenses

The paper’s central lesson is that the fix has to move from the model to the memory boundary, and from output inspection to provenance and causal attribution. Three defenses are evaluated. Counterfactual Composition Testing (CCT) re-runs the pipeline with individual memory entries removed to find which retrieved item is causally responsible for a violation; the paper reports it locates the causal entry with 87.5% accuracy and zero false alarms, against a forensics baseline that was blind across all 25 scenarios. Retrieval Concentration Monitoring watches for the tell-tale over-representation of a single injected document in what gets retrieved. Memory-Persistent Information-Flow Control (MP-IFC) carries trust labels along the write and retrieval paths so that content uploaded as untrusted data cannot silently re-emerge as trusted system context; the authors report it blocks 97% of attacks at the cross-session boundary where prior state-of-the-art fails on every informative case. They also argue a Retrieval-Coverage Dilemma: evading concentration monitoring structurally requires weakening the attack itself, which is what makes the approach resistant to the adaptive bypasses that defeated twelve earlier defenses. For teams shipping today, the practical takeaways are to treat every uploaded document as untrusted input that must never be promoted to system authority without an explicit provenance check, to log and label the source of every memory write, to add a causal-attribution step before concluding “the model did it,” and to segregate memory stores by tenant and trust level so a single upload cannot pollute everyone’s context. The researchers release an accompanying benchmark (an adversarial memory corpus with causal ground truth across financial and healthcare scenarios) so defenders can test their own attribution before an incident forces the question.

Status

This is a research finding about a class of agentic RAG systems, not a single vendor’s CVE. Figures below are drawn from the public preprint and its project page.

Item	Detail
Technique	Semantic Norm Drift (SND) — a policy-formatted document poisons shared agent memory and re-emerges as trusted context
Core claim	The Misattribution Gap: memory-poisoning artifacts are identical to model misalignment, so the model gets blamed and retrained while the attack persists
Attacker requirement	Ordinary document-upload access; no model access, no trigger, no repeated interaction
Reported detection evasion	Zero detections from four safety classifiers (one memory-poisoning-trained) across 510 checkpoints
Reported reasoning pattern	Agents cited the injected document as authority then complied in 59 of 65 valid entries
Proposed defenses	Counterfactual Composition Testing (87.5% causal-attribution accuracy), Retrieval Concentration Monitoring, Memory-Persistent Information-Flow Control (97% blocked at cross-session boundary)
Published	Preprint arXiv:2605.22842v1, 12 May 2026
Status	Research finding; mitigations are provenance-aware memory controls and causal attribution, not a patch