Demographic Parity vs Equalized Odds
Two of the most frequently cited fairness metrics in AI bias evaluation records — and why they cannot always be satisfied simultaneously.
Demographic Parity
Demographic parity (also called statistical parity) requires that the proportion of positive outcomes be equal across demographic groups. For example: if 30% of applications from group A are approved, 30% from group B should also be approved.
When to use: Appropriate when the base rate of the target variable is expected to be equal across groups by design — for example, synthetic datasets generated to represent balanced populations.
Equalized Odds
Equalized odds requires that both the true positive rate and the false positive rate be equal across groups. It conditions on the true outcome — groups with different base rates can still satisfy equalized odds if the error rates are consistent.
When to use: Appropriate when base rates legitimately differ across groups and the focus is on consistent accuracy rather than equal outcome rates. Common in medical diagnosis, credit risk, and fraud detection.
The Impossibility Result
When base rates differ across groups, it is mathematically impossible to simultaneously achieve demographic parity, equalized odds, and predictive parity (calibration). This is known as the fairness impossibility theorem. Bias evaluation records document which metrics were measured — organizations must determine which trade-offs are acceptable for their deployment context.
Regulatory Guidance
The EU AI Act does not specify which fairness metric must be satisfied. Article 10 requires documentation of data governance and bias examination procedures. NIST AI RMF similarly emphasizes documentation and organizational responsibility. Neither framework mandates a specific fairness criterion — leaving that determination to organizations based on their use case and risk tolerance.