Why ignoring sensitive Data doesn’t make AI fair
What happens when artificial intelligence makes decisions that affect millions of people, but the data it learns from carries the weight of history?
At the Hi! PARIS Summer School 2025, Solenne Gaucher, Assistant Professor at École polytechnique, explored one of the most pressing challenges in AI today: ensuring fairness in algorithms. Drawing on her research at the intersection of mathematics, computer science, and ethics, she highlighted why algorithmic bias is not just a technical problem, and not just an ethical one either.
Her central message: fairness in AI sits at the crossroads of society and mathematics, and addressing it requires contributions from both.
Key Takeaways
- AI systems are trained on historical datasets, which inevitably carry biases, stereotypes, and prejudices.
- These models risk amplifying inequalities when deployed at scale, making fairness a critical societal concern.
- Two misconceptions dominate: that fairness is only a technical issue, or only an ethical one, both are wrong.
- Competing notions of fairness exist, from correcting inequalities to ensuring equal treatment, which can sometimes be in tension.
- Sensitive attributes like gender or ethnicity cannot simply be “ignored”; paradoxically, ensuring fairness often requires explicit access to them.
Beyond ethics or mathematics alone
For Solenne, a key obstacle lies in the misconceptions surrounding algorithmic bias. On one side, many assume it is solely the responsibility of those building the algorithms, engineers, mathematicians, or computer scientists. Yet, as she emphasized, fairness has ethical implications that must be debated by society as a whole, especially by those most impacted by AI-driven decisions.
On the other side, some argue that fairness is purely an ethical question and offers little room for mathematical contribution. This too is misguided. Technical challenges in defining and operationalizing fairness remain open, and new methods are still needed.
The reality, Solenne explained, is that fairness in AI cannot be neatly divided: it requires ethical reflection and mathematical innovation, working hand in hand.
Conflicting definitions of fairness
Fairness itself is not a single concept. Depending on the context, it can mean very different things. Quotas, for example, may be considered fair in one setting because they correct for historical inequalities or underrepresentation. But in another setting, they may risk setting individuals up for failure, which could itself be seen as unfair in the long run.
This tension shows why fairness cannot be treated as a universal formula. Instead, it must be adapted to context, with careful attention to both the intended consequences and unintended consequences.
Solenne Gaucher at the Hi! PARIS Summer School 2025
Why ignoring sensitive attributes doesn’t work
A frequent misconception is that fairness can be achieved by simply excluding sensitive variables such as gender or ethnicity from a dataset. In practice, this is ineffective. With enough data, these attributes can often be inferred indirectly.
Conversely, to ensure that algorithms treat protected groups fairly, one must first be able to identify who belongs to these groups. This means that fairness sometimes requires, rather than avoids, the use of sensitive information.
A call for the next generation of researchers
Looking to the future, Gaucher’s advice to students and young researchers was clear: “Go for it.” Fairness in AI is not only a deeply important societal issue, but also a young and dynamic field. It is full of open problems, both practical and theoretical, waiting to be solved.
For those drawn to the challenge, the opportunity is twofold: to advance research, and to make AI systems more just and trustworthy for society.