Need to Know: contextual-integrity query rewriting for LLM delegation

What is this?

On June 2, 2026, Xinyue Huang, Xiaochun Cao and Wenyuan Yang posted Need to Know: Contextual-Integrity-Grounded Query Rewriting for Privacy-Conscious LLM Delegation (arXiv:2606.04067, cs.CR / cs.AI). The paper tackles a privacy exposure that is easy to overlook because nothing “breaks”: every time a user delegates a task to a cloud-hosted LLM, their prompt routinely carries task non-essential sensitive disclosures alongside the content the task actually needs. The provider — and anyone with access to its logs — receives both.

The authors’ core argument is that the usual mitigation, type-based PII redaction, is the wrong tool. A redactor that masks anything shaped like a name, an address or a card number is context-agnostic: it simultaneously over-discloses untyped sensitive context (a medical detail, a relationship, a location implied in prose) and over-removes answer-bearing spans the model needed to do the job. The contribution is a reframing plus a benchmark and a trained rewriter — not an attack.

How it works

The paper recasts privacy-preserving query rewriting under Contextual Integrity (CI): information flow is appropriate only when it fits the norms of the context it was shared in. Translated into a rule for delegation, that becomes a single test — a span should be forwarded to the cloud model only if it is necessary for the task. Necessity, not data type, is the decision criterion.

To make that measurable, the authors build DelegateCI-Bench, described as the first task-based Contextual Integrity benchmark for privacy-conscious delegation. It comprises 3,167 samples combining synthetic data across 11 tasks and 20 task types, real user queries drawn from WildChat, and a medical challenge set dense with sensitive information.

# Conceptual decision, per span — descriptive, not runnable code.
# Source: arXiv:2606.04067 (Huang, Cao & Yang, 2026).

type-based redaction:   mask(span) if regex/NER says "PII"   # context-blind
contextual integrity:   forward(span) only if task_essential(span)
                        else suppress(span)                  # necessity-based

On top of the benchmark, they train a query rewriter with a CI-guided reinforcement learning framework that turns “essential” versus “non-essential sensitive” spans into verifiable optimization signals — the rewriter is rewarded for keeping task-critical information while stripping unnecessary disclosure. Reported result: the learned rewriter reaches the best privacy-utility tradeoff, up to +10.1 average utility over on-device baselines.

Why it matters

This is the everyday side of LLM data exposure. There is no malicious payload and no compromised model — just the volume of sensitive context that flows to third-party providers because a prompt is the most convenient place to dump everything you know. For regulated workflows (health, legal, HR) that flow is a compliance surface as much as a privacy one, and “we redact PII before sending” is a weaker control than it sounds: type-based redactors miss context that isn’t a recognized PII type, and break utility when they blank out the span that held the answer.

The CI framing is portable beyond this one rewriter. Any team building a gateway in front of a cloud model — a proxy, a DLP filter, an on-device pre-processor — is implicitly making forward/suppress decisions. Doing it by necessity rather than by data type is a cleaner mental model, and a benchmark like DelegateCI-Bench gives a way to check whether a given filter is actually preserving answers while cutting leakage, instead of trading one failure for the other.

Defenses

The paper is the defense, but the practitioner takeaways are concrete. Treat data minimization toward cloud LLMs as a necessity test, not a regex pass: ask whether each sensitive span changes the answer, and suppress it if it does not. Measure both halves of the tradeoff — privacy and utility — because a redactor that quietly removes answer-bearing text fails silently and users route around it. Where feasible, run the rewrite/minimization step on-device or inside your trust boundary so the suppression happens before anything leaves, and keep a held-out evaluation set (medical or otherwise high-sensitivity) to catch over-disclosure regressions.

Two honest caveats. A learned rewriter is itself a model and can mis-classify a span as non-essential and drop something that mattered, or keep something it should not — so this is a risk-reduction layer, not a guarantee, and high-stakes flows still warrant human review and least-privilege scoping of what the cloud model sees. And the +10.1 figure is a benchmark average from the authors; real-world gains depend on your task mix and how sensitive your queries actually are.

Status

Item	Reference	Date	Notes
Primary paper	arXiv:2606.04067 (Huang, Cao, Yang)	2026-06-02	cs.CR / cs.AI; v1
Benchmark	DelegateCI-Bench	2026-06	3,167 samples, 11 tasks, 20 task types, WildChat + medical set
Method	CI-guided reinforcement learning rewriter	2026-06	Best privacy-utility tradeoff, up to +10.1 avg utility vs on-device baselines
Framing	Contextual Integrity: forward a span only if task-essential	2026-06	Replaces context-agnostic type-based PII redaction

This is a research result, not a disclosed product vulnerability — there is nothing to patch. The actionable takeaway is architectural: decide what reaches a cloud LLM by necessity, not by data type, and measure utility alongside privacy so your minimization layer is not silently deleting the answer.