Evaluating the most extreme potential failures or abuses of an AI system to inform robust risk mitigation and contingency planning.
A stress-testing methodology where systems are subjected to theoretical or simulated maximum-impact scenarios - adversarial attacks, cascading failures, regulatory violations - to quantify potential losses (financial, reputational, safety) and develop contingency plans. Governance mandates that high-risk AI applications undergo worst-case analysis annually, with documented response playbooks and executive review of findings.
A healthcare-AI vendor conducts worst-case analysis on its triage chatbot: they simulate simultaneous server failures, model misclassifications of critical symptoms, and data-breach scenarios to estimate patient-harm and develop response protocols, including Fallback Hotline activation and emergency patch-deployment procedures.
