Bad Algorithms & The Ethical Matrix

Wednesday 15:30-16:15

Main Stage

Algorithms can embed bias, they can propagate or even exacerbate inequality, or they can just be plain inaccurate. How do we keep track of all the potential problems? How do we make sure the algorithms we build “work well”? What do we even mean by that? In her talk at SHIFT, Cathy O’Neil will introduce the ethical matrix, a construction borrowed from moral philosophy, as a way of organizing our thoughts around important and urgent questions like these.

 

Cathy is the author of Weapons of Math Destruction,New York Times Bestseller and longlisted for the National Book Award

 

We live in the age of the algorithm. Increasingly, the decisions that affect our lives—where we go to school, whether we get a car loan, how much we pay for health insurance—are being made not by humans, but by mathematical models. In theory, this should lead to greater fairness: Everyone is judged according to the same rules, and bias is eliminated.

 

But as Cathy O’Neil reveals in this urgent and necessary book, the opposite is true. The models being used today are opaque, unregulated, and uncontestable, even when they’re wrong. Most troubling, they reinforce discrimination: If a poor student can’t get a loan because a lending model deems him too risky (by virtue of his zip code), he’s then cut off from the kind of education that could pull him out of poverty, and a vicious spiral ensues. Models are propping up the lucky and punishing the downtrodden, creating a “toxic cocktail for democracy.” Welcome to the dark side of Big Data.

 

Tracing the arc of a person’s life, O’Neil exposes the black box models that shape our future, both as individuals and as a society. These “weapons of math destruction” score teachers and students, sort résumés, grant (or deny) loans, evaluate workers, target voters, set parole, and monitor our health.

 

O’Neil calls on modelers to take more responsibility for their algorithms and on policy makers to regulate their use. But in the end, it’s up to us to become more savvy about the models that govern our lives. O’Neil empowers us to ask the tough questions, uncover the truth, and demand change.

 

 

 

Yes, take me to Cathy!

Speaker

CATHY O’NEIL

ORCAA , Founder