Self-hosted LLM infrastructure: securing model endpoints
You install vLLM or llama.cpp-server, start it, and your model is suddenly reachable over the network. Most inference servers ship with no authentication at all: any client that can reach the port can send prompts, and on many stacks the same port exposes weights, health checks, and debug endpoints. The model endpoint is your new perimeter, and by default it comes up open. This article is about the layers that close it: network placement, authentication, TLS, and logs.
The default state of an inference server
Inference servers are built for throughput, not access control. vLLM, Text Generation Inference, and llama.cpp-server bind to an interface you choose, and unless you configure an API key explicitly, requests are accepted as-is. The OpenAI-compatible API surface makes this worse: tooling assumes an Authorization: Bearer header is honored, but nothing enforces it out of the box. Assume the absence of auth is the default, and add controls in layers rather than relying on any single one.
Network placement before authentication
The cheapest control is not exposing the port at all. Bind the server to 127.0.0.1 and put a reverse proxy in front of it, or reach it over a WireGuard or Tailscale network so only your machines can route to it. Then firewall everything else: management interfaces, metrics endpoints such as /metrics, and anything that does not need to be public. A health endpoint can stay public if you run uptime checks; the rest should not. If a model server must be reachable from the internet, the proxy is the only thing that should face it.
Authenticating and terminating TLS
At the proxy, require an API key on every request and reject anything without one. Fail closed: an invalid or missing key should produce an error, not a degraded mode that silently serves traffic. Keep keys out of command lines, because argv is visible in process listings and shell history; pass tokens through environment variables or a keyring instead. Terminate TLS at the proxy and enforce HTTPS end to end. If you want to inspect or filter the traffic itself — for example to catch secrets in prompts or responses — the proxy has to terminate TLS and re-encrypt upstream, which is what HTTPS MITM means in practice. Rate-limit per key so a single leaked credential does not buy unlimited inference on your GPU.
Log hygiene at the endpoint
Inference servers log prompts, completions, and request metadata by default, and proxies log request lines. API keys can end up in those logs, either in headers or in URLs. Decide what the proxy records before you ship it. Two-layer redaction works well: first scrub known credential values, then apply pattern-based rules — keyword filters, regular expressions, and entropy scoring for strings that look random — and mask or block matches. Keep an audit trail of decisions (which key was injected, which request was denied, which rule redacted a value) rather than recording payloads, and make that log append-only with restrictive permissions so rotation and review stay safe.
Putting it together
For self-hosted agents, this is exactly the shape of trustless serve: a DLP reverse proxy for any OpenAI-compatible endpoint with per-host header and query injection, HTTPS MITM, and an append-only audit log that records events such as proxy.inject, proxy.deny, and dlp.redact without ever storing token values. The source is at https://github.com/ikkun1222/trustless, and the command reference lives in the docs at https://trustless-security.com/docs/.
The components are ordinary: bind localhost, authenticate at the proxy, terminate TLS where you can inspect, and log decisions instead of secrets. The question is whether your endpoint has all four. When did you last check who can reach your model server — and what it logs when they do?