LLM prompt leakage: what your model API requests reveal

Your AI agent reads files, runs commands, and talks to services all day. Everything it reads becomes eligible context for the next prompt — and every prompt becomes a request to a model API on somebody else’s machine. Prompt leakage is what happens when the request body contains more than you intended to send.

What’s actually in a request

A model API request is a JSON body: system prompt, message history, tool outputs, and often raw file contents. There is no “should this leave the machine?” check between context and request. If the agent read a config file with a hardcoded API key, that key can ride along in the conversation history verbatim. If a tool printed an env var dump, the dump goes in.

The leaky categories, in practice:

• Credentials. Keys pasted into configs years ago, tokens in debug output, .env values the agent read for context.
• Provenance-bearing data. Private code with internal hostnames, customer data the agent summarized, internal endpoints it discovered.
• Prompt-injection payloads. A webpage or document the agent read can contain instructions to exfiltrate context — and the model may oblige by placing secrets into its own next request.

What the provider sees

From the API provider’s side, each request is a record: the full prompt, the model, the tool calls, the timestamps. Depending on the provider’s retention and logging policies, that record can persist long after your session ends. Some providers log request bodies for abuse monitoring; some keep them for training. You rarely know which, and the terms change over time.

That’s the uncomfortable part of prompt leakage: it’s not a crash that alerts you. It’s a steady transfer of your context into someone else’s logs, with no signal at your end.

The metadata leak

The prompt is not the only thing in the request. Model name, tool definitions, system prompt, and traffic patterns — what time, what volume, from what host — are all visible too. For most teams the prompt is the critical part, but metadata alone can reveal which internal tools you use and when your agents run.

What to do about it

Leakage is a property of the architecture, so the fixes are structural:

1. Keep secrets out of context. A credential broker injects secrets at the subprocess boundary, so the agent never reads them in the first place. If the agent never saw the key, the key can’t leak into a prompt.
2. Scan outbound traffic. A DLP proxy inspects each request before it leaves and masks secret-shaped strings. Even when context hygiene fails, the key doesn’t cross the network boundary.
3. Audit what’s leaving. Record redactions and denials as events, so a leak attempt is visible instead of silent.

The residual risk

Even with all three layers, assume some leakage. Models can paraphrase, encode (base64, URL-encoding), or smuggle secrets through indirect channels. A DLP layer that handles the common encodings — including base64 and URL-encoded variants — covers the predictable cases, but the honest position is defense in depth, not elimination.

trustless approaches this from the credential side: a zero-dependency Go CLI that keeps secrets out of agent context — trustless run injects secrets into the subprocess environment and sanitizes stdout and stderr, including base64 and URL-encoded forms — and trustless serve, a DLP proxy that scans OpenAI-compatible traffic, redacts matches, and writes audit events. The command reference (/docs/commands/) shows how the pieces fit together.

The test: if you dumped your last 100 model API requests, could you swear none of them contained something you’d rather not have left the building — and can you even dump them?