The Go race detector: finding data races in CLI tools

The bug report says “crashes sometimes.” It never reproduces on the developer’s machine, fails intermittently in CI, and the stack trace points somewhere harmless. If you have ever chased one of those, you were probably looking at a data race: two goroutines touching the same memory without synchronization. In concurrent Go code, races are the classic works-in-testing, breaks-in-production failure: they depend on timing your test run happened not to hit.

Go ships a tool that finds these directly. The race detector is built into the toolchain, costs one flag to enable, and turns “sometimes crashes” into a specific, debuggable report.

What a data race is

A data race happens when two goroutines access the same memory location, at least one of them writes, and there is no synchronization between them. A classic example:

var count int
go func() { count++ }()
count++

Both goroutines read and write count with no lock, channel, or other ordering. The result is undefined — the increment can be lost. Sometimes the program still prints the right answer, which is exactly why these bugs survive testing.

Why CLIs are vulnerable

CLI tools look sequential but rarely are. Signal handlers, progress reporting, background workers, context cancellation, and embedded servers all introduce concurrency. A credential tool that proxies HTTP traffic or spawns subprocesses has goroutines running in parallel with the main flow, and every shared cache or counter is a race candidate. The moment a CLI grows any concurrency, it inherits that whole class of bugs.

How the detector works

The race detector is a runtime instrumentation built on the ThreadSanitizer approach. Compile with -race and every memory access is instrumented to record what happened, in which goroutine, and what synchronization exists. It maintains a shadow memory of recent accesses; when it observes two unsynchronized accesses to the same location with at least one write, it reports a race. Because it works at runtime, it only finds races that actually execute — which is why it is a detector, not a prover.

Running it

Two commands cover most needs:

go test -race ./...
go build -race ./cmd/you

go test -race is the workhorse — the test suite exercises code paths that may never run in your manual usage. Expect a slowdown of roughly 2x to 20x and higher memory use, so the instrumented build is for testing, not production. Run it in CI on every commit; races are cheap to find when fresh, expensive when they surface months later.

Reading the report

When the detector fires, it prints WARNING: DATA RACE, then a precise account: the exact address, the write or read in the “previous goroutine” and the conflicting access in the “current goroutine,” each with a full stack trace. The two stacks are the gift — no guessing which code paths collided; the detector names both. Fix with synchronization: a mutex around the shared state, or restructure so each piece of state is owned by one goroutine.

What it won’t catch

The detector finds races that execute; it cannot see races that don’t happen to trigger. It does not cover cgo or assembly code, and it says nothing about deadlocks. To make it effective, your tests must exercise the concurrent paths — run tests repeatedly (-count=10), stress the server and subprocess paths, and keep the suite race-enabled.

That discipline makes the flag meaningful. The test suite of trustless, a credential broker CLI for AI agents that injects secrets into subprocesses and proxies API traffic, runs 321 tests with -race — because for a tool handling credentials, a concurrency bug is not a quality issue, it is a security issue.

When did you last run your own test suite with the race detector — and what did it find that plain tests had been hiding?