Prompt instructions aren't an enforcement layer for enterprise agents

What is this?

On July 9, 2026, a preprint titled “From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agents” was posted to arXiv (2607.08028). It asks a question that sits underneath most enterprise agent deployments: when an LLM application starts as a prompt-driven prototype and then has to ship with real requirements — source boundaries, entity routing, answer contracts, reproducible traces — which of those guarantees can you actually enforce by writing them into the prompt, and which have to be enforced by code around the model?

The paper’s answer is blunt. Guarantees that live only in prompt instructions leak: violations reach the reader. The same guarantees, moved into a code-owned enforcement layer around the model, hold. This is an architecture finding, not an exploit — but it has direct security consequences for any team relying on system prompts to keep an agent inside the lines.

How it works

The authors describe a harness: deterministic behavior is pulled out of the prompt and into code — manifests, schemas, and validation artifacts — arranged around a composition boundary, the replaceable point where the LLM actually composes an answer. Source-backed claims stay the authority for runtime answers, but the rules about how an answer may be shaped (grounding, routing, formatting, what language is allowed) become machine-checkable contracts rather than polite requests in a prompt.

They instantiate this on a public-data slice of five corporate groups (25 listed companies) and test five contracts: source-grounding, entity-routing, trace, output-hygiene, and recommendation-language. A fault-injection control deliberately breaks each contract to confirm the validators actually fire, rather than passing everything by default.

The core experiment holds the model fixed and varies only the enforcement layer:

Enforcement layer         Trace-leak / recommendation-language contract
------------------------  ------------------------------------------------
Prompt instructions only  Violations reach the reader
External bolt-on guard    Blocks violations, but over-refuses (utility 88/120)
Code-owned harness        Blocks violations, full utility (120/120)

No attack payloads are needed to make the point, and none are reproduced here. The finding is about where a control lives, not about a clever string.

Why it matters

Two numbers carry the paper. First, holding the model fixed, prompt instructions alone let recommendation-language and internal-trace-leakage violations reach the reader, while the code-owned harness blocked them entirely. For an enterprise agent, an internal-trace leak can expose reasoning, retrieved context, or system details a user was never meant to see, and prohibited “recommendation” phrasing can be a genuine compliance problem in regulated settings. Neither is fixable by adding more prompt text.

Second, the contracts held under model substitution: across three hosted models, the harness checks passed on all 270 composition-boundary runs, and where the model-composed side failed, the failures were caught and recorded rather than silently shipped. That robustness matters because teams swap backbones constantly, and a guarantee that quietly depends on one model’s behavior isn’t a guarantee.

The over-refusal result is the honest counterweight. A bolt-on external guardrail also blocked the violations, but it dropped utility to 88/120 where the harness kept the full 120/120 — the familiar tax of a broad filter that refuses too much. Only scoped, code-owned enforcement preserved both safety and usefulness.

This lands squarely on a lesson the industry keeps relearning. Microsoft’s security team, writing in May 2026, put it plainly: an LLM is not a security boundary, and any tool parameter the model can influence must be treated as attacker-controlled. The design-patterns literature on securing agents against prompt injection makes the same structural argument — put deterministic controls around the model, don’t ask the model to police itself.

Defenses

Concrete takeaways for teams shipping enterprise LLM agents:

1. Treat prompts as behavior shaping, not enforcement. Any guarantee that must hold — source grounding, no internal-trace leakage, output format, allowed language — belongs in code: schemas and validators at the composition boundary, not sentences in a system prompt.

2. Put a validated boundary between model output and the user. Check every model-composed answer against machine-checkable contracts before release, and fail closed when a contract is violated.

3. Make enforcement model-independent. If swapping the backbone changes whether a contract holds, the contract isn’t really enforced. Re-run your contract checks across every model substitution.

4. Prefer scoped code-owned checks over broad bolt-on guardrails. External filters buy safety at the cost of over-refusal. Contract-specific validators preserve utility while still blocking the violation.

5. Version and archive your control artifacts. Manifests, schemas, validators, and traces should be versioned like code so an audit can reconstruct exactly why a given answer was allowed to reach a reader.

6. Add fault-injection tests. Deliberately break each contract in a test and confirm the validator flags it. A guardrail you never watched fail is a guardrail you can’t trust.

Status

Item	Reference	Date	Notes
Harness-engineering preprint	arXiv:2607.08028	2026-07-09	Code-owned contracts vs. prompt-only enforcement
Model-substitution check	Paper	2026-07-09	270/270 composition-boundary runs passed, 3 hosted models
Prompt-only enforcement	Paper	2026-07-09	Trace-leak and recommendation-language violations reach the reader
Harness vs. external guard	Paper	2026-07-09	Harness full utility (120/120); bolt-on guard over-refuses (88/120)
Industry context	Microsoft Security	2026-05-07	”Your LLM is not a security boundary”

This is a single case study on a public financial-disclosure slice, with a reference implementation and evaluation artifacts archived by the authors — it awaits broader, independent replication. But the direction is worth internalizing now: as agents move deeper into regulated enterprise workflows, the guarantees that survive are the ones you can point to in code, not the ones you asked the model to remember.