Jacobian Matrix
In AI explainability, the matrix of all first-order partial derivatives of a model’s outputs w.r.t. its inputs, used to assess sensitivity and feature importance.
Definition
A mathematical tool that quantifies how slight changes in each input dimension affect each output dimension—in neural networks, it can be used to compute saliency maps or measure local sensitivity. Governance uses Jacobian-based metrics to identify vulnerabilities (e.g., adversarial sensitivity) and to generate explanations highlighting which input features most influence a given decision.
Real-World Example
In a medical-imaging AI, engineers compute the Jacobian for each pixel with respect to the “tumor” probability. High-magnitude regions on the saliency map reveal which image areas drive the diagnosis, helping radiologists verify that the model focuses on plausible anatomical features rather than artifacts.
.svg)
