Building a DLP pattern layer: keyword pre-filter, RE2, and entropy

Your AI agent sends data to LLM APIs all day. Some of that data is code, some is configuration — and some of it is a secret that should never leave your machine.

The obvious defense is a DLP (data loss prevention) proxy: scan every outbound request, and if a key appears, block or mask it. Simple in theory. The hard part is knowing what a secret looks like without also flagging half your legitimate traffic.

This is the story of building that pattern layer, and the false-positive war that came with it.

Why known-value matching isn’t enough

The first instinct is substring matching: “if the request contains a value I have in my credential store, redact it.” That’s precise — zero false positives, because you’re matching exact values. It’s also fundamentally incomplete.

It only catches secrets you already know about. A teammate’s key that never went through your store, a token pasted into a config file years ago, a key that rotated after your last scan — known-value matching is blind to all of them. For a DLP layer on agent traffic, “mostly works for values I remembered to register” is not a security posture.

You need pattern detection: recognize the shape of a secret, not just its value.

The three-stage pipeline

Matching arbitrary strings against “does this look like a key” sounds like an entropy problem, and it is — but running entropy analysis on every byte of every request is too slow and too noisy. The workable design is a funnel:

1. Keyword pre-filter. Cheap string scan for high-signal fragments: sk-, AKIA, Bearer , eyJ (JWTs), BEGIN PRIVATE KEY, and friends. This rejects ~99% of traffic instantly, so the expensive stages only run on suspicious lines.

2. RE2 regex. Once a keyword hits, the line is checked against format regexes — the actual API key shapes. Rather than inventing these, the rule set is derived from gitleaks, the battle-tested secret scanner, and bundled into the binary (40 rules, MIT-licensed with attribution). Reusing a maintained rule set beats hand-writing regexes that miss real-world formats.

3. Shannon entropy threshold. Regex says “this matches the shape of an AWS key,” but the string also has to be high-entropy enough to actually be a key (default threshold 3.5, overridable per rule). This kills the classic false positive: AKIAIOSFODNN7EXAMPLE — the documentation’s placeholder key, which is low-entropy and would otherwise trip every rule.

The false-positive war

Pattern detection’s curse is noise, and the first real-world run proved it. The generic-api-key rule — the catch-all for key=value style secrets — is the worst offender, firing on half the legitimate configuration snippets an agent legitimately sends.

That’s why the config exposes pattern_disabled: a list of rule IDs to turn off, per installation. ["generic-api-key"] silences the noisy catch-all without touching the specific rules that matter. You can also point rules_file at your own gitleaks-compatible TOML and replace the bundled set entirely.

Two modes: watch, then act

Pattern hits in log mode are detected and counted but the body passes through unchanged — perfect for day one, when you want to see what the rules fire on without breaking anything. In mask mode, matches are redacted in place. The sane rollout is: log for a week, review the hits, disable the noisy rules, then flip to mask.

One operational detail worth stealing: configuration is reloaded on SIGHUP (plus a periodic refresh), and the pattern set is swapped atomically — a bad rule file fails the reload and keeps the previous state. A DLP layer that stops the world because of a typo is worse than no DLP layer.

The trade-off nobody talks about

Pattern DLP is a probabilistic filter: it will never catch everything (that’s what the known-value layer is for), and it will occasionally redact something innocent. The engineering question is whether you can live with that error instead of the other one — a key quietly leaving your machine in a prompt that looked harmless.

I built this as part of trustless, a credential broker CLI for AI agents (Go, zero dependencies, single binary). The DLP layer sits in trustless serve, a reverse proxy for any OpenAI-compatible endpoint — the full pipeline above, 40 gitleaks-derived rules, hot reload, audit events for every hit. 321 tests with -race, and every release is cosign-signed with an SBOM.

Curious what false-positive rules you’ve had to silence in your own secret scanning — and whether entropy thresholds ever let a real key through.