Stopping sensitive data from leaking into third-party LLM chats

What is this?

Every time an employee pastes a customer record, an API key, or a chunk of internal source code into a public chatbot, sensitive data crosses an organisational boundary that no firewall was watching. This is the risk catalogued as Sensitive Information Disclosure (LLM02) in the OWASP Top 10 for LLM Applications — and it is driven less by attackers than by ordinary productivity pressure. Static “please don’t paste secrets” warnings produce alert fatigue; blanket bans push people to shadow tools.

A paper posted to arXiv on 9 July 2026, Multi-Agent Firewall Architecture for Privacy Protection of Sensitive Data in Interactions with Language Models (2607.08282), by Hugo García Cuesta, Pablo Mateo Torrejón and Alfonso Sánchez-Macián of Universidad Carlos III de Madrid, proposes a defensive answer: a client-side firewall that inspects what a user is about to send to a third-party model and stops sensitive content before it leaves the machine. The system, Minos Verdict Mesh, is released as open source (MIT). It is a data-leakage-prevention study, not an attack technique.

How it works

The design separates interception from analysis. Two sensors capture outbound traffic. A browser extension (Chromium, Manifest V3) hooks into the chat interface of each site through a per-platform adapter, catching text and file uploads before they reach the page. A transparent proxy (built on mitmproxy) covers programmatic use, intercepting both HTTP(S) requests and persistent WebSocket sessions to known LLM providers; to read TLS-encrypted bodies it installs its own certificate authority in the local trust store, and it isolates user-authored content by filtering on the message role field.

Captured content is sent to a local backend (FastAPI) that runs a multi-agent pipeline orchestrated as a directed acyclic graph in LangGraph. The topology is deliberately static: the authors prototyped an LLM-driven router but found it slower, costlier and non-deterministic, so a fixed graph with conditional edges won out. Detection runs in tiers to keep the common case cheap. A first layer runs three detectors in parallel — deterministic pattern matching (regex with proximity keyword windows and checksum validation, so a 16-digit string is only flagged as a card number when card-related words sit nearby), zero-shot named-entity recognition using GLiNER, and a fuzzy code-similarity check (RapidFuzz over indexed Git repositories) that catches proprietary code even when it has been renamed or lightly refactored. Only ambiguous cases escalate to a second layer: an LLM reasons over an already-anonymised copy of the prompt, so cleartext secrets are never shipped to the analysis model.

That sanitisation-first order matters. Deterministic and NER hits are redacted into typed placeholders before any model sees the text, which resolves the awkward paradox of a privacy tool that itself leaks the data it inspects. A weighted risk score then aggregates findings — several low-risk items can together cross a threshold — and maps to a per-request policy of allow, warn, or block. In the browser, a block is a human-in-the-loop modal offering cancel, override, or send-an-auto-sanitised-version; in the proxy, enforcement is automatic (HTTP 403, or a dropped WebSocket message). Both fail closed if the backend is unreachable.

On the Nemotron-PII benchmark (US locale, 500 cases), the best configuration — a fine-tuned local Gemma 3 4B — reached 94.93% F1 (precision and recall both ≈94.9%), while a latency-first setting kept sub-second responses at ~82% F1. Multimodal, code-similarity and anonymisation-fidelity paths were disabled for that benchmark and remain unmeasured.

Why it matters

Most LLM security work watches what comes out of a model — jailbreaks, injected instructions, unsafe responses. This watches what goes in, and does so on the user’s side of the trust boundary rather than by routing traffic through yet another vendor. That closes a real gap: SDK-level guardrails require changing the application’s source, and cloud inspection services shift the very data you are trying to protect to a third party. A local-first, application-agnostic interceptor sidesteps both. The layered detector design is also a useful template — deterministic rules for the obvious cases, a small local model only for the ambiguous ones — for anyone building egress controls around AI tools.

Defenses

For teams worried about data walking out through chatbots:

• Treat prompts as an egress channel. Sensitive data leaving for a third-party LLM deserves the same DLP posture as email or file uploads; inspect it at the point of send, not after.
• Sanitise before you inspect. Redact deterministic hits into placeholders before any secondary model reads the text, so the safety layer never becomes the leak.
• Cover both surfaces. Browser chat and programmatic API/WebSocket traffic are different channels; a text-only or web-only filter leaves the other wide open.
• Keep detection local and fail closed. On-device analysis avoids exporting the data you are guarding; block rather than pass when the checker is unavailable.
• Read the numbers as a starting point. Results come from one English-locale PII dataset with code-similarity and multimodal paths switched off; validate against your own data before trusting a threshold.

Status

Item	Detail
Finding	Client-side, application-agnostic firewall that blocks PII, secrets and proprietary code from reaching third-party LLMs, mapped to OWASP LLM02
Source	Multi-Agent Firewall Architecture for Privacy Protection of Sensitive Data in Interactions with Language Models, García Cuesta, Mateo Torrejón, Sánchez-Macián, Universidad Carlos III de Madrid, arXiv, 9 July 2026
System	Minos Verdict Mesh — browser extension + mitmproxy sensors, LangGraph DAG pipeline (deterministic + GLiNER NER + code similarity + conditional LLM); open source, MIT
Key result	94.93% F1 on Nemotron-PII (fine-tuned local Gemma 3 4B); ~82% F1 sub-second in latency-first mode
Type	Defensive research / data-leakage-prevention tool (preprint; no CVE)