Fairness Metrics Explained
A technical reference for the fairness metrics documented in CertifiedData bias evaluation records. Each metric measures a different aspect of distributional disparity — no single metric captures all dimensions of fairness.
Demographic Parity Difference
Measures whether the proportion of positive outcomes is equal across demographic groups. A value of 0 indicates identical positive outcome rates. A non-zero value indicates disparity — it does not by itself indicate unfairness or legal violation.
Equalized Odds Difference
Measures whether true positive rates and false positive rates are equal across groups. Equalized odds is harder to satisfy simultaneously with demographic parity when base rates differ across groups.
False Positive Rate Disparity
Tracks whether individuals who should receive a negative outcome are incorrectly flagged at different rates across groups. Relevant in high-stakes applications such as fraud detection and criminal risk scoring.
Predictive Parity Gap
Measures whether individuals with the same predicted score have equal probability of the actual outcome across groups. Also called calibration difference.
Class Imbalance Ratio
Documents the proportion of positive to negative class instances in the dataset. High imbalance can lead to models that systematically underperform on minority classes — but the ratio itself is a documentation metric, not a fairness verdict.
No combination of fairness metrics can simultaneously be satisfied in all deployment conditions. When base rates differ across demographic groups, it is mathematically impossible to achieve demographic parity, equalized odds, and predictive parity at the same time. Bias evaluation records document which metrics were measured and what values were found — they do not prescribe which combination is acceptable for a given deployment.
Organizations deploying AI systems must determine which fairness criteria are appropriate for their use case based on legal requirements, domain norms, and organizational risk tolerance. CertifiedData records support that determination by providing traceable measurement documentation.