Have a project in mind? Let's talk.

AI agent incident response is not ordinary outage response with a new label. When an agent fails, it can keep acting: spending budget, calling tools, sending messages, and touching production systems while your team is still deciding whether something is wrong. Microsoft's Agent Governance Toolkit publishes a concrete incident classification and response workflow for agent systems, and it is the most usable public runbook structure we have seen. This guide turns that workflow, plus AWS reliability guidance, into a practical runbook you can adapt this week. Source: AI Agent Incident Classification and Response Workflow, Microsoft Agent Governance Toolkit
The toolkit starts with severity levels, each with a response target. P0, critical, means the agent caused or is actively causing harm, with an immediate response inside one hour; examples include unauthorized financial transactions and PII breaches. P1 covers detected policy bypasses with harm potential, such as a kill switch failure or a successful prompt injection, with a four-hour target. P2 is an anomaly without confirmed harm, like unexpected tool calls or audit log gaps, inside 24 hours. P3 is a governance configuration issue with no user impact, handled within a week. Source: AI Agent Incident Classification and Response Workflow, Microsoft Agent Governance Toolkit
Alongside severity, the toolkit assigns a category drawn from the OWASP Agentic Top 10: HIJACK for adversarial takeover, CAPABILITY_BREACH for unauthorized actions, DATA_LEAK for exposed sensitive data, TRUST_FAILURE for identity or trust-scoring failures, CASCADE for multi-agent failure propagation, AUDIT_FAILURE for compromised trails, RESOURCE_ABUSE for excessive consumption, BIAS_HARM for discriminatory outcomes, and POLICY_BYPASS for circumvented deterministic rules. OWASP published its agentic list precisely because autonomous, goal-driven agents create security failure modes that chatbot-era categories did not cover. The category matters because it decides who gets paged: a DATA_LEAK pulls in privacy counsel, while a RESOURCE_ABUSE stays an engineering problem. Source: OWASP Top 10 for Agentic Applications, OWASP GenAI Security Project
The toolkit's workflow runs triage, contain, investigate, remediate, notify, post-mortem. Triage has a 15-minute target: classify severity and category, and assign an incident commander. Containment follows immediately for P0 and P1. Investigation pulls audit logs and reproduces the behavior with evidence before anyone touches a fix. Remediation updates the policy or code, deploys the governance change, and verifies it with tests. Notification covers affected individuals, regulators where required, and internal stakeholders. Two ordering choices in that workflow are deliberate and worth copying. Investigation comes before remediation, because a fix applied before the root cause is understood usually just hides the failure until it recurs with a different signature. And verification is part of remediation, not an afterthought: the updated governance is deployed and then proven with tests before the incident moves on. The post-mortem closes the loop by updating the workflow itself when the incident exposed a gap in it. Source: AI Agent Incident Classification and Response Workflow, Microsoft Agent Governance Toolkit
For a P0, the toolkit prescribes an immediate kill switch activation, isolating the agent from all network communication, preserving audit logs with an explicit instruction not to rotate or delete them, and notifying the incident commander and legal within one hour. A P1 means disabling the affected policy rule or tool capability, switching the agent to read-only mode where available, and reviewing the last 24 hours of audit logs to scope the damage. P2 and P3 incidents become tracked issues with reproduction steps and a policy fix scheduled into the next sprint. The pattern to copy is that containment is pre-decided per severity: nobody should be negotiating whether to pull the kill switch during a P0. Source: AI Agent Incident Classification and Response Workflow, Microsoft Agent Governance Toolkit
The toolkit lists the artifacts to collect and preserve for every incident: audit log entries for the agent, the policy decision chain showing which rules matched, the agent's identity and trust scores at the time of the incident, OpenTelemetry policy-evaluation spans retained for 90 days by default, approval decisions, and session state snapshots. Several artifacts carry retention of the investigation duration plus three years. Two implications are worth acting on before any incident: your audit logging has to exist and be tamper-resistant, and your retention settings have to be chosen now, because you cannot extend them retroactively once lawyers are involved. Source: AI Agent Incident Classification and Response Workflow, Microsoft Agent Governance Toolkit
The toolkit maps notification to specific regimes, and it aligns its workflow with the Colorado AI Act, EU AI Act Article 62, and the NIST AI RMF MANAGE function. Its summary: Colorado's AI Act expects notice to affected consumers within 90 days of discovering a harmful consequential decision, describing the system, the decision, and how to contest it. The EU AI Act's serious-incident regime expects reports without undue delay and no later than 15 days after becoming aware, to the Member State's market surveillance authority, for qualifying serious incidents. Internally, it recommends notifying stakeholders within 24 hours for P0 and P1, the security team immediately for any policy bypass, and customers per contractual SLA. Treat these as the toolkit's summary of fast-moving rules, and confirm your own obligations with counsel rather than reading a blog post as legal advice. Source: AI Agent Incident Classification and Response Workflow, Microsoft Agent Governance Toolkit
Incident response assumes incidents, and AWS's Well-Architected guidance for agentic workloads covers the failure mode that generates many of them: the long-running, stuck workflow. AWS recommends controls that detect and terminate unexpected long-running executions, rating the risk of skipping this as high. Its guidance includes timeouts on the external compute an agent calls, sized to include the maximum process time plus edge-case latency such as cold starts, timeouts at the agent layer for sessions waiting on user input, dead letter queues for timed-out processes, monitoring of execution times and timeout frequency with alerts on repeats, and graceful termination plus cleanup routines for recovery. It also advises that agent-layer error handling tell users clearly what happened and what to do next, without exposing sensitive internal detail. Source: Implement timeout mechanisms on agentic workflows, AWS Well-Architected Generative AI Lens
Our assessment is that most teams running agents in production have monitoring, some have kill switches, and very few have a classified, pre-decided response path with an incident commander and per-severity containment. The classification step is the one that pays off fastest, because it converts an ambiguous 2 a.m. alert into a named severity with a clock and an owner. This runbook connects directly to controls we have covered before: circuit breakers bound the blast radius during containment, and the observability checklist is what makes the investigation step possible at all.
The Microsoft toolkit frames incidents by category: adversarial takeover, unauthorized actions, data leaks, trust failures, multi-agent cascades, audit failures, resource abuse, biased harm, and policy bypasses, mapped to the OWASP Agentic Top 10. Source: AI Agent Incident Classification and Response Workflow, Microsoft Agent Governance Toolkit
The toolkit's targets are immediate, under one hour, for P0 incidents where the agent is causing harm; under four hours for P1 policy bypasses; under 24 hours for P2 anomalies; and within a week for P3 configuration issues. Triage itself has a 15-minute target. Source: AI Agent Incident Classification and Response Workflow, Microsoft Agent Governance Toolkit
Activate the kill switch immediately, isolate the agent from network communication, preserve audit logs without rotating or deleting them, and notify the incident commander and legal within one hour. Source: AI Agent Incident Classification and Response Workflow, Microsoft Agent Governance Toolkit
They can. The toolkit points to Colorado's AI Act, with 90-day consumer notice for harmful consequential decisions, and the EU AI Act's serious-incident reporting, due without undue delay and no later than 15 days for qualifying incidents. Confirm your obligations with counsel. Source: AI Agent Incident Classification and Response Workflow, Microsoft Agent Governance Toolkit
AWS recommends timeout mechanisms at the function, workflow, and agent-session levels, dead letter queues for timed-out processes, execution-time monitoring with alerts on repeated timeouts, and graceful termination with cleanup routines. Source: Implement timeout mechanisms on agentic workflows, AWS Well-Architected Generative AI Lens
Ready to take the first step towards unlocking opportunities, realizing goals, and embracing innovation? We're here and eager to connect.