Incident Response

Investigation phase

1. Correlate findings from sub-agents:
   • Timeline: when did the deployment happen vs. when did symptoms start?
   • Scope: is the issue isolated to one deployment or cluster-wide?
   • Cause: does the health report explain the symptoms?
2. If root cause is unclear, expand the search:
   • Check other deployments in the same namespace
   • Check namespace resource quotas
   • Check cluster-level events

Response phase

1. If the issue is caused by a bad deployment:
   • Delegate to deploy agent to rollback
   • Delegate to healthcheck agent to verify recovery
2. If the issue is infrastructure-related:
   • Report the infrastructure issue with evidence
   • Suggest manual remediation steps

Incident summary format

INCIDENT SUMMARY
================
Affected: <service(s)> in <namespace(s)>
Duration: <start> to <end/ongoing>
Severity: <P1/P2/P3>
Root Cause: <one-line summary>

TIMELINE:
- <timestamp>: <event>
- <timestamp>: <event>

ROOT CAUSE ANALYSIS:
<detailed explanation>

ACTIONS TAKEN:
- <action and result>

STATUS: <Resolved|Mitigated|Investigating>

Validation Criteria

• Sub-agents are used appropriately (don't do their job manually)
• Findings are correlated into a coherent timeline
• Root cause is identified with evidence
• If rollback is needed, it is performed and verified
• Incident summary is complete and accurate

Example

Input: "Our nginx app in the demo namespace is returning 503s after the last deploy"

Expected agent delegation:

1. Delegate to healthcheck agent -> "Check health of nginx in demo namespace"
2. Delegate to deploy agent -> "Show rollout history for nginx in demo namespace"
3. Correlate findings
4. If bad deploy: delegate to deploy agent -> "Rollback nginx in demo namespace"
5. Delegate to healthcheck agent -> "Verify nginx in demo is healthy after rollback"

Expected output: A complete incident summary with timeline, root cause, and resolution status.