Browser agents leak their model identity through how they click

What is this?

Known By Their Actions: Fingerprinting LLM Browser Agents via UI Traces (arXiv 2605.14786, posted 14 May 2026) shows that you can tell which large language model is driving a browser agent simply by watching how it interacts with a web page. The authors collect the agent’s on-page actions — navigation, clicks, scrolls, typing, the order and timing of element interactions — and train lightweight classifiers on those traces. Across 14 frontier models they report agent-identification scores of up to 96% Macro F1.

The key point is that this fingerprint does not depend on anything the agent operator can easily change. It is not the User-Agent string, the HTTP headers, the IP address, or the TLS stack — all of which can be spoofed. It is the behaviour: the temporal and structural dynamics of how a given model decides where to look, what to click, and in what sequence. The agent’s actions are, in effect, a signature of the model that produced them.

How it works

The researchers instrument target pages with injected JavaScript that records the sequence of UI events an agent generates while completing a task. Each model produces a recognisable pattern — preferred interaction order, characteristic dwell and inter-action timing, how it recovers from a failed click, how it scans a form. These patterns are stable enough that a simple classifier trained on labelled traces can attribute an unseen session to its source model with high accuracy.

Crucially, the method is model-agnostic on the collection side. The paper releases a labelled corpus of agent interaction traces across four web environments and a browser harness compatible with both closed and open-source models, so the attack works whether the agent is built on a proprietary API or a self-hosted open-weight model. No privileged access to the agent is required — only an ordinary web page the agent visits, plus client-side telemetry that any site already collects for analytics or bot detection.

agent performs task on page
   → injected JS logs UI action trace (clicks, scrolls, timing, order)
      → classifier trained on labelled traces
         → predicts underlying model  (up to ~96% Macro F1, 14 models)

Why it matters

Behavioural fingerprinting turns a property most teams treat as invisible — which model is behind an agent — into something a remote website can read. That has several consequences. It defeats attempts to keep an agent’s model choice private or to rotate between providers without being noticed. It hands a reconnaissance primitive to attackers: once a site knows the exact model, it can serve indirect prompt injection or jailbreak payloads tailored to that model’s known weaknesses, raising success rates. It also fuels the bot-detection and access-control arms race, letting sites selectively block, throttle, or deceive specific agents regardless of header spoofing. And for users who run agents on sensitive workflows, a behavioural signature is a deanonymisation channel that survives the usual network-level countermeasures. It echoes a broader line of work on inferring LLM usage from side channels, such as traffic-fingerprint analysis of agent interactions (arXiv 2510.07176).

Defenses

There is no single fix, because the leak is intrinsic to how each model behaves. Practical mitigations focus on reducing signal and raising the cost of attribution:

• Treat model identity as leakable. Threat-model the assumption that a remote site can learn your agent’s model from behaviour alone, and do not rely on header or IP obfuscation for anonymity.
• Add controlled randomisation. Introduce jitter into inter-action timing and, where safe, vary interaction order so that temporal/structural patterns are less separable. This trades a little efficiency for less distinctiveness.
• Normalise the action layer. Route agent decisions through a deterministic execution harness that standardises how clicks, scrolls, and form fills are emitted, so different models share a common low-level interaction profile.
• Isolate high-risk pages. Because fingerprinting enables model-tailored injection, keep untrusted web content away from agents that hold credentials or tools, consistent with lethal-trifecta and least-privilege guidance.
• Monitor and rate-limit collection-side telemetry. On the defender’s own properties, recognise that fine-grained UI telemetry is dual-use and can itself profile visiting agents.

These reduce reliability but, per the paper’s framing, do not eliminate behavioural attribution — distinctiveness is a feature of model decision-making, not a bug to be patched once.

Status

The work is a research disclosure, not a product vulnerability, so there is no CVE or vendor patch. The authors position it as enabling reproducible research into behavioural attribution of LLM agents, and release their dataset and harness for that purpose. Defenders building or deploying browser and computer-use agents should fold model-identity leakage into their threat models now, before agent fleets scale further.

Key date: paper posted 14 May 2026 (arXiv 2605.14786).