RAG pipeline security: secrets in embeddings and vector stores

You pointed a RAG pipeline at your internal docs last quarter: chunk everything, embed it, answer from your own material. What you built, without deciding to, was a second copy of every document you indexed — retrievable by similarity and backed up with the rest of your infrastructure.

The copy is the problem. Most pipelines store each source chunk verbatim next to its vector so answers can quote it, and that makes the store a plaintext archive, not an abstraction. A README with an example API key, a committed config file, an exported chat log — each is now a searchable record that did not exist before. Secrets that were sitting harmlessly in one place are suddenly in two, and one of the two was built for fast retrieval.

Where the pipeline collects secrets

Ingestion does not filter. Chunkers split on headings and token counts, and text splitting has no opinion about whether a line looks like a credential. Most RAG corpora pick up secrets the same way documentation picks up dust: wiki pages with copy-paste tokens, runbooks with real-looking keys, test fixtures.

Two structural details make it worse. Chunking splits secrets: a token spanning a chunk boundary is still embedded and still retrieved as two halves of one answer. Ingestion is not idempotent either — re-running it after an edit does not remove the earlier copy unless you implement deletion yourself.

The vector store is a plaintext store

Calling it an “embedding store” understates what is on disk. For quoting and source attribution the chunk text has to be retrievable, so the store holds the same bytes as the original document. That is retention, not compression. Deletion does not rescue you either: removing a document needs an explicit delete call, backups keep their own copies, and the embeddings alone act as a membership oracle — a nearest-neighbor query can confirm that a given string was in the corpus even after the text is gone.

Retrieval re-exposes everything

The leak does not stop at the store. Every query that matches a secret-bearing chunk assembles that chunk back into the prompt, and the prompt goes to a model API — the same downstream path as any other model traffic, with server-side caches and provider-side logging on top. Poisoned documents are a separate attack; the secret-shaped leak needs no attacker, because your own pages do the leaking.

Gate at ingestion, not at retrieval

Everything downstream of the chunker is a copy, so the point of control sits between the source documents and the chunker:

1. Scrub the corpus before indexing. Run known-value and pattern-based redaction over every file you will ingest — rules derived from gitleaks plus Shannon-entropy detection for tokens no rule knows. A dry-run report shows what would be redacted before anything is written.
2. Fix the document, not the store. If a source file contains a credential, remove it from the source; scrubbing the copy while the original still leaks is theater.
3. Treat the vector database as sensitive at rest. Permissions, backups, and retention should match your .env files, because the store is a plaintext copy of a slice of your documentation.
4. Scan the retrieval path. A DLP proxy in front of the model API redacts secret-shaped strings in each request, catching what slipped past ingestion.

One implementation of the scrub-before-index pattern is trustless: trustless dlp scrub-text and scrub-db apply two-layer redaction over text files and SQLite databases with a dry-run default, and trustless serve scans OpenAI-compatible traffic at the retrieval boundary. See the command reference.

A scrub pass over a corpus before indexing costs minutes. Purging one secret from every vector-store backup and index snapshot costs days — and only after someone notices the leak. When did you last query your own vector store for strings that look like an API key, and what did it return?