Running local LLMs: why you still need DLP for agent traffic
“I run Ollama locally, so nothing leaves my machine — I do not need DLP.”
It is the common objection to DLP for agent traffic, and it misses the point. DLP for agent traffic was never about where the model runs. It is about where secrets go before and after the model sees them. Running the model on your laptop changes the endpoint, not the pipeline.
What a local gateway actually sees
Local inference servers — Ollama, llama.cpp, vLLM — expose OpenAI-compatible
endpoints on localhost. Your agent is pointed at http://127.0.0.1:11434 or
similar, and every request it sends contains the full context: system
prompt, tool definitions, file contents, and — crucially — tool output.
If a tool returned a secret — an API key in a build log, a credential in a
config dump — that value is now in a request to the model, local or not.
Two more realities of local gateways:
- Localhost is not authenticated. Any process on the machine can call the endpoint; a compromised tool or malicious MCP server can read the full context of every request.
- Local servers log too. Request and response logs on disk hold whatever the agent sent, secrets included.
Why “local” does not mean “safe”
The leak paths that matter do not depend on the model’s location:
- The context window persists. A secret that enters context ends up in session transcripts, summaries, and replay files — on your disk, and in provider request logs if you ever switch to a hosted model.
- Local models can leak the secret back. An instruction-following local model asked to “repeat the API key from earlier” may well comply. The redaction point has to be upstream of the model, not downstream of your trust in it.
- The pipeline is shared. Most setups flip between local and hosted models for the same agent; a secret that flowed through the local gateway will flow the same way through the hosted one.
Redacting at the gateway
The fix is to scrub agent traffic at the point where it enters any OpenAI-compatible endpoint, local or remote, in both directions:
- Requests: redact known values and high-entropy strings before the model sees them, so the secret never enters context in the first place.
- Responses: redact completions that echo a credential back, so the agent never receives it.
- Detection layers: a keyword pre-filter catches obvious values cheaply;
regex rules catch formats like
sk-...; Shannon-entropy scoring catches keys no rule knows about. Known-value redaction handles the rest.
A DLP reverse proxy sits in front of the endpoint and applies the policy before forwarding
the request. trustless serve does exactly this for any OpenAI-compatible
endpoint — including the one on your localhost — with per-host header and
query injection, HTTPS MITM for encrypted client traffic, and log-or-mask
modes per rule.
Operational details that matter
Gateway DLP fails quietly unless you design for it: rules should hot-reload without restarting the proxy, a bad config should keep the previous ruleset instead of disabling protection, and scrubbing should be dry-run by default so you can see what would have been redacted before applying it. Patterns are only as good as their coverage: the bundled rule set derives from gitleaks’ rules, with attribution, and entropy detection is the safety net underneath.
Running the model locally is a real privacy improvement over sending everything to a third party — but it is not a substitute for controlling what enters the model’s context.
trustless is a single static binary that includes this DLP proxy plus retroactive scrubbing of existing databases, documented under /docs/commands/.
If your agent pointed at a local model today, could you say for certain what was in the last request it sent?