E6: Cathy O'Neil: Weapons of Math Destruction
We sat with Dr. O'Neil to discuss her book, Weapons of Math Destruction: How Big Data increases inequality and threatens democracy. Dr. O’Neil has a PhD in math from Harvard, and a posdoc from MIT. She is a seasoned columnist for Bloomberg view, and her blog https://mathbabe.org/.
The book was quite insightful. It gives us a raw perspective on the effects of the algorithms often used as decision making engines by both private and public institutions. From crime prediction software, to college rankings, predatory advertising, criminal justice procedures, and even candidate screenings; these algorithms operate on the background behind a curtain of anonymity, and safeguarded as proprietary models only accessible to those who capitalize on them. With little to no accountability over their effectiveness or accuracy.
O’Neil calls these types of algorithms WMDs or Weapons of Math Destruction. Every chapter covers a practical application, some of which mirror, reinforce, and even amplify our own society’s biases, ultimately having a damaging effect in people’s lives. O’Neil outlines the key characteristics of these types of models as follows: (Opacity) The lack of transparency of certain modes preventing us from understanding their operating logic. Thus, evaluating their efficacy and fairness. Their scale, reach, and whether if they are inadvertently or designed to work against users, encoding and capitalizing on prejudice and biases. I.e. predatory loan ads targeting lower income people, or for-profit college ads targeting vulnerable communities praying on their lack of opportunities and access to federal loans.
Ultimately, the author presents a compelling case on how these algorithms capitalize on a false promise of an internet meant to democratize opportunities and access, while turning every interaction into another data point and capitalizing on people’s vulnerabilities. That is truly the dark side of big data.
An extremely relevant read.