AI Bias and Fairness

Where Bias Comes From

AI bias is not a single problem — it enters systems at every stage of development. Training data bias occurs when the data used to train a model reflects historical inequities. An AI trained on decades of hiring decisions will learn that men were hired more often for engineering roles — not because men are better engineers, but because historical discrimination skewed the data. Amazon discovered this in 2018 when its AI recruiting tool systematically downgraded resumes containing the word 'women's' (as in 'women's chess club captain').

Representation bias occurs when certain groups are underrepresented in training data. Facial recognition systems trained predominantly on light-skinned faces perform significantly worse on dark-skinned faces. A landmark 2018 study by Joy Buolamwini and Timnit Gebru at MIT found that commercial facial recognition error rates for dark-skinned women were up to 34.7%, compared to 0.8% for light-skinned men.

Measurement bias happens when the variables used as proxies for a concept do not measure it accurately. Using arrest records as a proxy for criminal behavior, for example, reflects policing patterns rather than actual crime rates.

The Fairness Paradox

One of the most important insights in AI ethics is that different mathematical definitions of fairness are often mutually exclusive. Consider a predictive model used in criminal sentencing. You might want the model to have equal false positive rates across racial groups (so that it does not incorrectly flag more Black defendants as high-risk). You might also want equal prediction accuracy across groups (so that the model is equally reliable for all defendants). These two goals cannot both be fully satisfied when base rates differ between groups.

This is not a technical problem that better engineering can solve — it is a value judgment about which type of error matters more. Should the system minimize false positives (wrongly flagging innocent people) or false negatives (failing to flag people who will reoffend)? The answer depends on political, ethical, and social priorities that technical systems cannot resolve on their own.

This is why AI fairness requires democratic deliberation, not just better algorithms.

Approaches to Mitigating Bias

Audit and testing: Regular bias audits using disaggregated metrics — breaking down performance by race, gender, age, geography, and other dimensions — reveal disparities that aggregate metrics hide. A model with 95% overall accuracy might have 99% accuracy for one group and 80% for another.

Diverse development teams: Homogeneous teams are less likely to anticipate how a system will affect communities they do not represent. Teams that include people from varied backgrounds are more likely to catch harmful assumptions during design.

Participatory design: Involving affected communities in the design of AI systems — not just as test subjects but as stakeholders with decision-making power — produces systems that better serve diverse populations. This is especially important for systems deployed in education, healthcare, and criminal justice.

Regulatory requirements: The EU AI Act mandates bias testing for high-risk systems. New York City's Local Law 144 requires annual bias audits for AI hiring tools. These regulations create accountability that voluntary commitments alone have not achieved.