Catching rogue agents by reading their activations, not their messages

What is this?

On July 7, 2026, a preprint titled “When Agents Go Rogue: Activation-Based Detection of Malicious Behaviors in Multi-Agent Systems” was posted to arXiv (2607.06807). It targets a blind spot in how we currently defend LLM-based multi-agent systems (MAS): most defenses read the text agents exchange, and assume attacks are semantically obvious and that the system runs in neat, synchronized rounds. Real deployments break both assumptions.

The authors propose AcMAS, a detection framework that ignores the surface messages and instead inspects each agent’s internal activation states — the hidden-layer representations the model produces while it reasons. Their claim is that a compromised agent “looks” different on the inside even when its output is crafted to look clean, and that this signal survives the asynchronous timing of real MAS execution where graph-based propagation models fall apart.

How it works

A multi-agent system is a set of LLM agents that pass messages, call tools, and share memory to complete a task. The dominant defensive approach models this as a graph: nodes are agents, edges are interactions, and you trace how a malicious instruction propagates from node to node. That works when the attack is loud (a visible “ignore your instructions” injection) and when messages arrive in lockstep so the graph is temporally coherent.

AcMAS drops both requirements. For each local agent, it extracts activation-space features from the model’s internal states as the agent processes its inputs, then scores those features to decide whether the agent is behaving normally or has been co-opted. Because the signal lives inside a single agent’s forward pass, it does not need the global interaction graph and it does not need every agent to be on the same clock — the paper frames this as synchronization-robust detection.

The framework is evaluated against three attack families that map cleanly onto the real MAS threat surface:

Attack family            What the adversary does
-----------------------  --------------------------------------------------
Stealthy prompt          Plants instructions phrased to read as normal task
injection                content, evading text-level classifiers
Tool output manipulation Feeds a tampered tool/API result back into the
                         agent loop as trusted context
Memory poisoning         Writes a crafted entry into shared/long-term memory
                         so it resurfaces and steers later decisions

The second design choice is what happens after detection. State-of-the-art defenses tend to isolate a flagged agent — cut it out of the system — which also destroys whatever useful work it was doing and can stall the task. AcMAS instead uses the same activation signals to attempt activation-level correction, nudging the compromised agent’s internal state back toward normal so it can keep contributing. No exploit payloads are needed to understand the idea, and none are reproduced here.

Why it matters

The reported numbers are the headline. Against stealthy attacks, AcMAS beats graph-based baselines by +0.22 F1 in synchronous settings (0.94 vs. 0.72) and by +0.55 F1 in asynchronous settings (0.93 vs. 0.38). That async gap is the interesting part: it quantifies how badly graph-propagation defenses degrade when the system is not running in tidy rounds — dropping to 0.38 F1, barely better than a coin flip — while the activation-based detector holds at 0.93. Most production agent systems are asynchronous, so this is the regime that actually matters.

It also lands in the middle of an industry shift. Microsoft’s security team spent June 2026 warning that agents “moving from reading to acting” expands the blast radius of any single compromised agent, and the broader agentic-security literature keeps finding that message-level filtering is porous. Activation-based inspection is a different bet: instead of trying to catch every possible malicious phrasing, watch the mechanism the model uses to comply.

The catch is honest and stated in the paper: reading activations requires access to model internals, i.e. open-weight backbones you host yourself. If your MAS is built on closed API models, you cannot see the activations, and this defense does not apply as-is. That makes AcMAS most relevant to teams running self-hosted open-weight agents — a growing but not universal deployment shape.

Defenses

Concrete takeaways for teams running multi-agent systems:

1. Don’t rely on text-level filtering alone. Stealthy injection and tool-output manipulation are specifically designed to pass message inspection. Treat inter-agent messages and tool results as untrusted input, and add a second detection layer that does not depend on how the payload is worded.

2. If you self-host open-weight models, instrument the activations. The paper shows internal-state monitoring generalizes across backbones, attack intensity, and system scale. Capturing hidden-state features per agent is a viable telemetry source you likely aren’t collecting today.

3. Prefer correction or quarantine-with-review over hard isolation. Cutting a flagged agent can break the task and hand the attacker a denial-of-service win. Design your response so a suspected-compromised agent is contained but recoverable, not simply killed.

4. Test defenses under asynchronous execution. A defense that scores 0.94 F1 in synchronous benchmarks may collapse to 0.38 in the async conditions your system actually runs in. Red-team with realistic, out-of-order agent timing before trusting any propagation-based control.

5. Threat-model the three vectors explicitly. Stealthy prompt injection, tool output manipulation, and memory poisoning are distinct entry points with distinct owners (input handling, tool integration, memory store). Assign each a control rather than assuming one guardrail covers all three.

Status

Item	Reference	Date	Notes
AcMAS preprint	arXiv:2607.06807	2026-07-07	Activation-based MAS detection + correction
Reported detection (sync)	Paper	2026-07-07	0.94 F1 vs. 0.72 graph baseline on stealthy attacks
Reported detection (async)	Paper	2026-07-07	0.93 F1 vs. 0.38 graph baseline
Key limitation	Paper	2026-07-07	Requires open-weight backbones (activation access)
Industry context	Microsoft Security	2026-06-30	”Reading to acting” agent risk expansion

This is early research, not a shipping product — the results are the authors’ own and await independent replication. But the direction is worth tracking: as agent systems get more asynchronous and attacks get quieter, defenses that read a model’s internal state may outlast defenses that only read its words.