Opacity
The absence of transparency in how an AI model arrives at decisions or predictions, posing challenges for trust and regulatory compliance.
Definition
Refers to “black-box” models whose internal representations and decision logic are inaccessible or too complex for human interpretation. Opacity raises issues in high-stake domains where explainability is required. Governance may limit opaque models to low-risk tasks or mandate pairing with external explainability tools and human-review processes to mitigate opacity’s trust and compliance risks.
Real-World Example
A financial regulator prohibits the use of deep-ensemble fraud models without explainability. A bank therefore restricts such opaque models to internal analytics and uses a simpler, interpretable model with LIME explanations for customer-facing fraud alerts—ensuring compliance with transparency requirements.
.svg)
