Prompt caching and secrets: where your context goes to sleep

Your context window does not disappear when the session ends. If the model API you route your agent through has prompt caching — automatic at some major providers, an opt-in flag at others — the stable opening of your conversation is copied into a server-side cache.

That cache is why your bill is lower and responses feel faster. It is also where secrets go to sleep: a credential that enters the cached prefix is no longer just something your agent saw — the provider now stores it.

What prompt caching actually does

Prompt caching is a cost and latency optimization. When a request arrives, the provider checks whether its opening tokens match a recently stored prefix. On a hit it reads the cached tokens, charges a fraction of normal input, and answers faster. On a miss it processes everything and stores the new prefix.

The cache is keyed on exact text and shared across requests from the same account — cheap, and therefore storage. Every token in a repeating prefix, your system prompt, your tool definitions, the early turns of a session, is copied into infrastructure you do not administer and kept for minutes to hours.

Secrets make themselves at home in the cached prefix

Where do secrets actually sit in agent traffic? Keys pasted into system prompts. Config files the agent reads early. Tool output that echoed an environment variable. All of it lands in the stable opening of the conversation — exactly the part caching stores.

The consequences compound:

• A secret in the cached prefix outlives the session. Clearing the chat or scrubbing your local store does not touch the provider’s cache — your cleanup stops at your disk.
• Your own later requests replay it. A session that reuses the same prefix re-reads the secret from cache, keeping it alive after you rotated it.
• The cache sits on top of whatever logging the provider already does. Persistence is layered, and you control none of it.

Cache hits are a timing oracle

A cache hit returns faster than a miss, and the cache key is exact text. That latency is an observable signal: researchers have shown that a request containing a candidate string returns measurably faster when that string was recently cached, letting an attacker test whether a target’s traffic contained it. Cache scoping per organization or project closes the obvious version, but the tension is structural: the more broadly a provider shares cache keys to maximize hits, the more the cache acts as a shared oracle.

Prompt-injected content sharpens the problem. Text an agent pulls from a webpage is cached verbatim for the window — attacker-chosen data with a server-side TTL, written by your own requests.

Keep the secret out before it can be cached

The answer is unchanged, and caching is why it must happen before the request, not after:

1. Inject credentials at the process boundary. A credential broker hands the secret to a subprocess environment at spawn time, so the value never appears in a prompt and never enters a cache.
2. Scan outbound model traffic. A DLP proxy masks secret-shaped strings in each request before it leaves. What is redacted in flight cannot be persisted server-side.
3. Treat the model API as a durable store. If you would not write a secret to a shared file that lives for hours, do not put it in a prompt.

The first layer is what trustless implements: trustless run resolves secrets from an existing pass or Bitwarden store and injects them at spawn while sanitizing output, and trustless serve is a DLP reverse proxy that scans and redacts OpenAI-compatible traffic before it leaves — details in the command documentation.

The question worth sitting with: if the provider’s cache were ever exposed, what is the oldest secret currently sleeping in your cached prefixes?