When agents rewrite themselves: why self-evolution makes every attack lineage-persistent

What is this?

Most agent-security analysis assumes a fixed target: the model weights, the memory store and the tool set do not change between deployments, so defenders can enumerate the interfaces and guard them. A systematization paper published on arXiv on 22 June 2026, Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studies (arXiv 2606.23075), argues that this assumption breaks the moment an agent can modify itself — updating its own parameters, memory, tools or even architecture without human intervention.

The paper’s central claim is that self-evolution is not just one more feature to secure. It changes the nature of every existing threat: a compromise that used to last one session becomes encoded across generations, and the agent’s own improvement loop becomes the infrastructure that carries the attack forward.

How it works

The authors organize the analysis around what they call the Module–Lifecycle Attack Surface (MLAS) matrix: five functional modules (Brain, Cognitive Resource, Execution, Self-Design, Collective) crossed with five lifecycle stages (Bootstrap, Propose, Evaluate, Commit, Serve), giving 25 cells. For each cell they describe the exposed interfaces and how a known attack behaves there.

Static agent                          Self-evolving agent
-----------------------------------   -------------------------------------------
Injection corrupts ONE session        Injection is written to memory / distilled
                                      into weights / saved as a new tool
Reset clears the compromise           No reset: the change persists across cycles
One exploit -> one effect             Exploits self-reinforce through the
                                      evaluate/commit loop and propagate
Weights + tools = immutable anchors   Every component is mutable, including the
                                      defense and verification logic itself

Three shifts drive this. Transient becomes persistent, because an injected instruction can be baked into long-term memory or a fine-tune. Single-point becomes self-propagating, because a poisoned memory entry can corrupt the selection signal used for the next training round, which then admits more poison. And the agent becomes both target and vector: reward hacking during self-play can produce misalignment that the training loop actively selects for, and in multi-agent populations a single compromised agent can spread through knowledge sharing.

Of the 25 cells, the paper rates 17 as critical with no effective defense, 7 as high where current defenses are insufficient, and only 1 as partially mitigated. It formalizes seven amplification effects — including generational accumulation, Lamarckian propagation, a capability ratchet that resists removal, and an optimizer–optimizee collapse where the component meant to check evolution is itself evolved. In a comparative case study of two open-source frameworks, an evolution-native design activated 3.5× more attack-surface cells than an evolution-augmented one and showed a reported 100% persistence rate across 40 payloads, while its co-located security scanner blocked only about 2.5% of attacks on the evolution pathway.

Why it matters

This reframes a class of systems that is moving quickly from research demos into products. If self-modification is on the roadmap, then the familiar defensive posture — filter the input, sandbox the tool, constrain the output at deployment — is structurally incomplete, because it assumes anchors that no longer exist. The finding that a scanner sitting next to an evolution loop catches almost nothing is the concrete warning: bolting a static check onto a self-modifying pipeline does not cover the pathway that actually changes the agent.

The honest caveat: this is a systematization and a two-framework case study, not a survey of production incidents, and it deliberately studies designs built to evolve aggressively. The 100% persistence figure describes those specific frameworks and payloads, not agents in general. The value is in the map, not in a universal severity number.

Defenses

The paper’s defensive principles line up with practical steps for anyone building or evaluating self-improving agents:

1. Keep a real immutable anchor. If every component is mutable, there is nothing to verify against. Hold core alignment constraints, evaluation logic and audit records outside the evolution loop, on infrastructure the agent cannot rewrite.
2. Guard the evolve pathway, not just the serve pathway. Placing a scanner at inference time misses the Propose/Evaluate/Commit stages where changes are introduced. Gate every self-proposed weight, memory, tool or architecture change behind review that the agent does not control.
3. Treat memory and self-generated training data as untrusted. Poisoned experience is the seed of the amplification loop; validate provenance before it feeds the next generation, as with memory-poisoning and sleeper-memory defenses.
4. Monitor longitudinally. Session-scoped detection cannot see a threat that accumulates across generations. Compare behaviour across evolutionary cycles and alert on drift, not just on single bad outputs.
5. Prefer bounded evolution. Evolution-native designs activate far more attack surface than evolution-augmented ones. Constrain what may change, how often, and under what independent check.

Status

Item	Reference	Date	Notes
Self-evolving agent attack surface (MLAS matrix)	arXiv 2606.23075	2026-06	25 cells: 17 critical / 7 high / 1 partial; 7 amplification effects; 100% persistence in evolution-native case study
Sleeper / persistence attacks on agents	arXiv 2605.28201	2026-05	Plant–persist–trigger pattern the evolution loop can entrench
Long-term memory security survey	arXiv 2604.16548	2026-04	Lifecycle view of memory attacks and governance

The shift the paper documents is uncomfortable but clean: self-evolution converts every known attack from session-bounded to lineage-persistent, and static defenses were built for a world where the agent stays still. Until evolution-aware verification exists, the safest control is the oldest one — keep something the agent cannot change.