Reading an agent's tool-use intent before it acts: pre-action probes
A June 2026 paper reads two signals — is a tool needed, and how risky is it — straight from an agent's activations before execution, turning post-hoc logs into a pre-action oversight layer.
What is this?
On June 8, 2026, Hariom Tatsat and Ariye Shater (Barclays Quantitative Analytics) published Beyond the Black Box: Interpretability of Agentic AI Tool Use, a mechanistic-interpretability study with a defensive punchline: an agent’s decision to reach for a tool — and how consequential that tool action is — is often readable from the model’s internal state before the action happens.
Most agent observability is external and late. Prompts reveal correlations, evaluations score outputs, and logs only arrive after the model has already called the tool. The paper’s argument is that this is the wrong moment for security-relevant decisions. In long-horizon runs, an early wrong call reshapes every step that follows, so a monitor that reads intent at the decision boundary — before execution — buys an intervention window that log-based tooling never gets.
How it works
The method combines two long-standing interpretability tools. A Sparse Autoencoder (SAE) decomposes an agent’s dense activations into sparse, inspectable features; linear probes — lightweight logistic classifiers — then read specific signals out of those features. Immediately before each action, the framework pools the last 32 pre-action tokens’ hidden states, encodes them through public SAEs (GPT-OSS SAEs and Gemma Scope), and runs two readouts:
- a Tool-Need probe (binary: will the model call a tool at this step?), and
- a Tool-Risk probe (three tiers: low, medium, high) that estimates the consequence of the next tool action.
Risk here is about impact, not topic: low-risk actions are read-only lookups and retrieval, medium-risk are bounded writes, and high-risk are authentication, outbound communication, or code execution — think getusertoken, sendemail, or shell-exec. Tellingly, the features that most drive the Tool-Risk probe cluster around credential and authentication concepts, and the tool-decision signal concentrates in late transformer layers, close to where the model commits to answering directly or delegating.
The probes were trained only on NVIDIA’s Nemotron function-calling dataset and then applied to two open-weight models, GPT-OSS 20B and Gemma 3 27B, with the Berkeley Function Calling Leaderboard held back as a strict zero-shot transfer test.
Why it matters
For defenders, this reframes tool-call safety as something you can inspect proactively. A high-risk tool action — sending mail, minting a token, running a shell command — is exactly the kind of step that prompt injection and confused-deputy attacks abuse. A pre-action probe gives an oversight layer a chance to pause or require confirmation before that action fires, rather than discovering it in the logs afterward. It also generalizes: because the monitor reads internal representations rather than tool-specific output patterns, one probe framework can watch many tools and many steps.
The results are promising but honest. Tool-Need accuracy is 75.3% on GPT-OSS and 71.4% on Gemma; Tool-Risk reaches roughly 90% on tool-call rows, though its macro-F1 is dragged down because medium-risk cases are rare and hard to separate. In replay, the probe flagged about three-quarters of missed-tool cases — its strongest role is as an omission auditor. Under zero-shot transfer to a new benchmark, the tool-need signal survives (Probe-1 agreement with ground truth of 87.7% on GPT-OSS), but thresholds drift badly across tool namespaces, producing high false-alert rates until recalibrated.
Defenses
Treat these probes as an internal oversight layer, not a security boundary. Deployed well, the recipe is cheap and interpretable: capture activations selectively at high-value decision points, run the two readouts before dispatching a tool call, and gate the consequential actions — authentication, outbound comms, code execution — behind a confirmation step when the Tool-Risk probe fires high. Pair it with external controls (least-privilege tool scopes, human-in-the-loop for irreversible actions, execution logging) rather than replacing them.
Mind the limits the authors are candid about. Tool-Risk depends on a heuristic risk scheme and transfers less cleanly than Tool-Need, so thresholds must be recalibrated per tool distribution before you trust an alert. Only two open-weight backbones and a selected set of layers were studied, so portability across architectures and post-training recipes is unproven. And crucially, a probe reads the model’s own internal signal: it is an honesty-of-intent monitor, not an injection detector — an attacker who reshapes what the model intends can, in principle, move the signal too. Use it to catch missed and risky calls, not as a standalone guarantee.
Status
| Item | Detail |
|---|---|
| Publication | Preprint, June 8, 2026 (arXiv:2605.06890v3) |
| Authors | Hariom Tatsat, Ariye Shater (Barclays) |
| Models evaluated | GPT-OSS 20B, Gemma 3 27B (instruction-tuned) |
| Tool-Need accuracy | 75.3% (GPT-OSS) / 71.4% (Gemma) |
| Tool-Risk accuracy (tool rows) | 90.3% / 88.5% |
| Best use | Pre-action omission and risk auditing at the tool-call boundary |
The work builds on a growing line of activation-probing research — including probes that flag hallucinated tool selections and high-stakes interactions from hidden states — pointing toward internal, pre-action monitoring as a practical complement to the external evals and logs most agent stacks rely on today.