Incident response for AI agent compromises: a runbook
An agent you did not fully supervise just exfiltrated a credential, or executed a command you never intended. The model is not the problem; the problem is the access it held, the session it ran in, and the trail it left. Incident response for an agent compromise follows the same skeleton as any credential incident, with a few twists. This runbook assumes the worst has happened and moves through containment, evidence, and recovery in order.
Phase one: contain the session
Stop the agent before you investigate. Kill the agent process and its child processes — an agent can spawn shells, background jobs, and long-lived tool servers. On Linux, kill the process group, then check for children that survived. Revoke the session’s access: expire the agent’s OAuth tokens, rotate any credentials the session could have read, and disable the account or service identity it ran under. Do not skip rotation because the agent probably did not touch a given key. You cannot know yet, and rotation is cheap compared with the alternative.
Phase two: preserve evidence
Before you change anything else, copy what you need. Snapshot the audit log, shell history, and tool output from the session, and note the session’s exact time range from start to kill. If you use a credential broker, its audit log is the primary evidence: every credential resolution, injection, and denial carries a timestamp, which lets you reconstruct which secrets the session actually received. Preserve logs somewhere the compromised host cannot reach — if the attacker still has a foothold, evidence written back to the same machine is evidence lost.
Phase three: determine scope
Work backward from the evidence. Which credentials did the session resolve, and which of those were actually used? Check the audit log for denied actions as well as allowed ones; denials tell you what the attacker attempted. Correlate timestamps with network or SIEM logs to see where traffic went. The goal is a concrete list: credentials exposed, data accessed, commands executed. Where the evidence cannot support a claim, mark it as unknown rather than guessing.
Phase four: remediate and learn
Rotate every credential the session could have seen, not just the ones you know were used. If the broker supports token refresh, force re-authentication so refresh tokens are replaced. Then fix the cause. Was the agent given a credential it did not need? Did a prompt injection trigger a tool call with no human gate? Reduce standing access, add a confirmation step for sensitive commands, and tighten the DLP posture on the agent’s egress.
Why the audit log decides the outcome
The difference between a two-hour incident and a two-week incident is usually the same thing: a complete, tamper-evident audit trail. An agent compromise is only as understood as the log that recorded it. Tooling that logs agent actions without storing secret values — like trustless (https://github.com/ikkun1222/trustless), which records injections and denials in an append-only JSONL audit log — gives you the facts you need without creating a second leak.
Drill it before you need it
Run this runbook against a staging agent once a quarter. Time the phases, check that logs survive rotation, and verify that every credential you can imagine being exposed is actually rotatable. Runbooks fail during drills, not during incidents — that is the point of doing both.
Which phase of this runbook would your team be least able to execute right now, and what would you need to make it possible?