Task done, privacy leaked: agents over-share across tool calls
A June 2026 benchmark shows a tool-using agent can complete its task while quietly passing unnecessary private data to intermediate tools — success does not mean need-to-know disclosure.
What is this?
ToolPrivacyBench is a benchmark, published on arXiv (2606.28061) on June 26, 2026 by Shijing Hu, Liang Liu, Zhu Meng and Zhicheng Zhao, that measures a blind spot in how we evaluate tool-using LLM agents: whether an agent hands each tool only the private data that tool actually needs.
The gap it targets is specific. Conventional function-calling benchmarks score task completion and API correctness — did the agent call the right tools with valid arguments and finish the job? Privacy benchmarks, meanwhile, usually inspect the agent’s final answer or ask it to make a privacy judgment. Neither one watches the middle of a multi-tool run. ToolPrivacyBench does exactly that: it audits the whole executed trajectory and asks whether “task-private atoms” — discrete pieces of sensitive information — flowed only to the tools and downstream sinks that were authorized to receive them.
How it works
Each of the benchmark’s 2,150 cases is paired with a policy knowledge base describing which information is allowed to reach which tool or sink. Of those cases, 1,150 are fully synthetic privacy-sensitive business workflows and 1,000 are adapted from existing multi-tool and function-calling benchmarks.
The evaluation is trajectory-level rather than answer-level. The agent runs against mock business backends, and the system records two things: the actual arguments passed to each tool call and the backend audit logs. An evaluator then compares those recorded flows against the case’s policy knowledge base. If a private atom reached a tool that had no need for it, that is flagged as over-disclosure — regardless of whether the task itself succeeded. This is an evaluation harness, not an attack: it inspects normal agent behavior against a stated policy, so no exploit payload is involved.
The framing the authors formalize is a need-to-know disclosure boundary: each tool should receive only the information necessary for its stated purpose. Purpose-bound information flow, in other words — the same intuition behind contextual-integrity and least-privilege thinking, applied to the arguments an agent puts into intermediate tool calls.
Why it matters
The headline finding is uncomfortable and easy to overlook: across nine widely used agents, successful tool execution did not imply appropriate privacy disclosure. An agent could complete its assigned task correctly while transmitting unnecessary private information through intermediate tool calls along the way.
That decouples two things product teams tend to conflate. A green “task succeeded” signal says nothing about what the agent leaked to reach that result. In a real deployment, those intermediate tools are often third-party APIs, logging services, search backends, or downstream sub-agents — each an additional party that now holds data it never needed. A customer-support agent that resolves a ticket may still have pushed a full customer record into a lookup tool that only required an order number.
This is the quiet, non-adversarial cousin of prompt injection. No attacker is required; the over-sharing is the agent’s own default behavior under a plausible task. It matters because agents are increasingly wired into chains of tools where every hop is a potential sink, and because current evaluation practice — optimizing for completion — actively rewards agents that grab and forward more context than they should. A related June 2026 survey on privacy in LLM agents and earlier work on tool-orchestration leakage point the same way: the orchestration layer, not just the final response, is where private data escapes.
Defenses
ToolPrivacyBench is a measurement tool, but its need-to-know framing maps directly onto concrete mitigations for anyone shipping tool-using agents:
- Enforce need-to-know at the tool boundary. Define, per tool, exactly which fields it may receive, and strip or reject everything else before the call leaves the agent. Do not rely on the model to self-limit.
- Audit trajectories, not just answers. Log and review actual tool arguments and backend access, then compare them against an explicit disclosure policy. A task that “succeeded” can still fail a privacy audit.
- Minimize context forwarded to tools. Pass identifiers and scoped references instead of full records; resolve sensitive data as late and as narrowly as possible.
- Treat each tool and sub-agent as a distinct party. Apply data-flow controls between hops the way you would between services, including third-party APIs and logging pipelines.
- Add over-disclosure to pre-deployment evaluation. Benchmark privacy over-disclosure alongside task success, and re-run it after any change to tools, prompts, or model version.
Status
| Item | Detail |
|---|---|
| Artifact | ToolPrivacyBench, arXiv:2606.28061 |
| Published | June 26, 2026 |
| Type | Evaluation benchmark (defensive / measurement) |
| Scope | Purpose-bound privacy over-disclosure in tool-using agents |
| Size | 2,150 cases (1,150 synthetic workflows + 1,000 adapted) |
| Agents evaluated | Nine widely used agents |
| Key finding | Task success does not imply need-to-know disclosure |
FAQ
What is ToolPrivacyBench?
ToolPrivacyBench is a benchmark described in arXiv paper 2606.28061 (June 26, 2026) that measures whether tool-using LLM agents over-share private data during multi-tool tasks. It audits the full execution trajectory to check that sensitive information reaches only the tools authorized to receive it.
What does “purpose-bound privacy” mean here?
It means each tool should get only the information necessary for its stated purpose — a need-to-know disclosure boundary. Passing extra private data to a tool that does not require it counts as over-disclosure, even if the overall task completes successfully.
Why isn’t task completion enough to judge an agent as safe?
Because completion and disclosure are independent. The benchmark found that agents can finish a task correctly while still routing unnecessary private information through intermediate tool calls. A “success” signal does not tell you what the agent leaked to intermediate tools and sinks.
How is this different from prompt injection?
Prompt injection requires malicious content to hijack the agent. Over-disclosure here is non-adversarial: it is the agent’s own default behavior on a legitimate task, quietly forwarding more private data than needed. No attacker is involved.
How can developers reduce over-disclosure?
Enforce per-tool field allow-lists, forward scoped identifiers instead of full records, audit actual tool arguments against an explicit policy, and add over-disclosure to pre-deployment evaluation rather than only measuring task success.