Poisoning what a web agent remembers: triggered attacks on multimodal memory

What is this?

In June 2026, a paper titled MemVenom: Triggered Poisoning of Multimodal Memories in Web Agents (arXiv:2606.10742) described an attack against a design choice that has quietly become standard: giving a web agent a persistent memory. Modern web agents no longer treat each task as a blank slate. They observe page states — text and screenshots — and write those observations into an external memory store so that on a later task they can retrieve relevant past experience. That memory extends the agent’s effective context well beyond the model’s input window, and it measurably improves task performance. It also creates a new, durable attack surface, and that is what the paper targets.

The key difference from ordinary prompt injection is persistence. A malicious prompt is transient: it influences one turn and is gone. A poisoned memory entry, once written, can be retrieved again and again, shaping decisions across many future sessions. The paper studies memory that is multimodal and graph-structured — nodes hold text and image evidence, edges hold semantic relations — which is exactly the memory design increasingly used to make web agents robust.

How it works

This is a description of a published research technique, not an operational guide, and the details below are deliberately kept at the level the paper itself uses to frame the threat. There is no payload here to copy.

The attack is built as a small, modular malicious subgraph rather than a single crafted string, and it separates two problems that memory poisoning has to solve. The first is recall: getting the poisoned entry retrieved at the right moment. MemVenom optimizes a recall component so that a specific “trigger-bearing” observation — a particular page state the agent will later encounter — causes the agent’s retriever to pull the malicious memory back into context. The second problem is influence: once the malicious memory is in context, getting the agent to actually act on it rather than ignore it. For that, the technique adds a post-recall prioritization component that pushes the agent to follow the recalled content.

The design detail worth noting is modularity. The recall trigger and the prioritization component are optimized once, then assembled with a replaceable goal-bearing component that carries whatever the attacker wants the agent to do. Because the goal slot is swappable, a new objective can be dropped in without re-optimizing the trigger or the prioritization machinery. That is the uncomfortable part: the expensive work of getting reliably recalled and reliably obeyed is paid for once and reused, and the effect compounds because a single poisoned subgraph can influence a chain of downstream decisions rather than a single step.

Why it matters

Memory poisoning matters precisely because it decouples the moment of compromise from the moment of harm. The attacker plants content during one interaction; the damage happens later, during an unrelated task, when the trigger condition is met and the agent recalls the entry as if it were its own trustworthy experience. For a web agent with tools — one that can fill forms, click, send messages, or move data — recalled-but-poisoned memory is a plausible route from “read a malicious page once” to “take an attacker-chosen action next week.”

The multimodal angle widens the surface further. When evidence nodes include images and screenshots, the trigger need not be text a filter would flag; it can be a visual state of a page. And because memory is shared across an agent’s tasks, one write can affect many later sessions, which is the same one-to-many amplification that makes RAG knowledge-base poisoning worse than single-turn injection. Recent surveys of long-term memory security in LLM agents, such as arXiv:2604.16548, place exactly this class of attack — persistent, retrieval-triggered, cross-session — among the hardest to detect after the fact.

Defenses

Treat written memory as untrusted input, not as ground truth. The core mistake is letting an agent recall its own stored observations with more authority than it would grant a fresh web page. Memory read back from the store should be re-evaluated against current policy, not trusted because “the agent wrote it.”

Separate data from instruction inside memory. A recalled entry can describe what was seen without being allowed to command what to do next. Tag stored spans by origin and enforce that retrieved memory informs reasoning but cannot by itself authorize a side-effecting action.

Constrain and attribute writes. Limit what can be written to memory, from which sources, and under what conditions. Keep provenance on every entry — which task, which page, which observation produced it — so a later flagged action can be traced back to the entry that drove it. Post-hoc auditing tools for poisoned agent memory, such as MemAudit, rely on exactly this kind of attribution.

Guard the retrieval step, not just the input box. Because the attack works through recall, defenses that only inspect the user’s prompt never see it. Monitor which memory entries are retrieved and whether a specific recurring observation reliably pulls the same entry into context — a signature of an engineered recall trigger.

Scope and expire memory. Persistent, unbounded, globally shared memory is the worst case. Time-box entries, scope them to the task or session that created them, and require re-derivation of trust for anything that outlives its original context, so a single poisoned write cannot influence unrelated tasks indefinitely.

Status

Item	Reference	Date	Notes
Paper	arXiv:2606.10742	June 2026	Triggered poisoning of multimodal, graph-structured web-agent memory
Target	Memory-augmented web agents (text + image evidence nodes, semantic edges)	—	The memory design increasingly used to improve web-agent performance
Mechanism	Recall component + post-recall prioritization + replaceable goal component	—	Modular malicious subgraph; goal is swappable without re-optimization
Key property	Persistence and cross-step compounding	—	Compromise and harm are decoupled in time
Context	Memory-security survey arXiv:2604.16548; auditing MemAudit	2026	Retrieval-triggered, cross-session poisoning is hard to detect post hoc

MemVenom does not claim web-agent memory is unusable — it argues that persistent, retrievable memory needs the same suspicion we already apply to any other untrusted input. For teams shipping agents that remember, the practical takeaway is narrow and testable: the retrieval path and the write path are security boundaries, and an entry the agent authored is not automatically an entry the agent should trust.