LangChain and LlamaIndex credentials: framework-level secret handling

LangChain and LlamaIndex both read provider keys from environment variables by convention. Set OPENAI_API_KEY, import the library, and the first model call just works. That convenience is exactly why credentials end up in places they should not: committed .env files, notebook cells, and printed tracebacks. Understanding where the framework handles secrets — and where it stops — is the first step to keeping yours.

What the frameworks do with your keys

Both libraries look for well-known environment variables when you construct a model client without explicit credentials. The key then travels inside the client object into HTTP headers, retry loops, and error messages. If that object gets serialized, pickled, or logged, the key goes with it.

Recent framework features try to help. Secret-value wrappers redact themselves in string representations, and callback systems sanitize events before they reach tracing backends. These are genuine improvements, but they only cover the paths the framework knows about. The moment a key passes into a tool, an arbitrary function, or a custom chain step, you are outside the protection.

Where the gaps are

The gaps cluster in three places. First, tool code: any function you hand to an agent can print, return, or embed a credential, and the framework cannot intercept what it does not see. Second, traces and callbacks: observability integrations receive full input and output payloads, so a key that appears in a prompt or tool result gets recorded. Third, persistence: notebooks, saved chains, and cache files retain whatever was in memory, keys included.

The environment is the boundary

Both frameworks treat the process environment as the source of truth for credentials, which makes the environment the real security boundary — not the framework. Whatever you set at launch is what gets passed to HTTP clients, retry logic, and serialization code. That also means variable-name collisions are a real risk: one exported variable can be read by LangChain, LlamaIndex, and a dozen CLI tools at the same time, and a key meant for one service is suddenly available to all of them. Scope the environment to the process, and you scope the credentials with it.

Handling secrets at the framework level

Treat the framework as untrusted for secret handling and move credentials out of its reach. Set the environment variables at process launch from a store instead of letting a .env file sit next to your code. Keep secrets out of prompts and tool arguments entirely — an agent should reference a capability, not a token. And assume output hygiene is your job: anything the agent prints, returns, or writes should be considered potentially logged.

One practical pattern is launching the agent process with secrets injected into its environment and scrubbing everything it emits. A wrapper that spawns the subprocess, injects the variables, and sanitizes stdout and stderr — including base64 and URL-encoded variants of known values — closes the loop the frameworks leave open. That is what trustless run does: it resolves a key by name from your password store, injects it into the subprocess environment, and redacts any trace of it in output before you see it. The tool is a single static Go binary with no runtime dependencies, and the source is at https://github.com/ikkun1222/trustless.

Framework-level secret handling is real, but it stops at the framework’s boundary. The question worth asking before your next model call: if your agent printed its own API key right now, would anyone notice?