BOOK REVIEW | "WEAPONS OF MATH DESTRUCTION: HOW BIG DATA INCREASES INEQUALITY AND THREATENS DEMOCRACY" BY CATHY O'NEIL
photo & text by nacrowe
in my previous career as a teacher i witnessed the transition of evaluations from being mostly observational and face-to-face in nature by superiors to that of analytics based on student performance. at the time i found the process rather opaque and was left unaware in any open, transparent manner what the goals or metrics being utilized were and to what extent they impacted my score. i was not alone in this criticism and it has only gotten worse in years since.
WEAPONS OF MATH DESTRUCTION (BROADWAY BOOKS, 2016) by data scientist CATHY O'NEIL gets into the ethical pitfalls that are besieged on disadvantaged populations by algorithms that essentially maintain the hegemonic status quo of the power set. essentially algorithms have been utilized by corporations and government entities to make sense of impressive large amounts of information, known as BIG DATA, to make predictions about nearly every facet of our lives as individuals and effectively perpetuate inequality. these predictions are the bases of models that are used to define us in terms that effect how we are policed, how we are sentenced, at what rate we are sold insurance or business loans, how we are evaluated in school and so on. basically there are examples abound in this book about the pernicious feedback loops that essentially create self-perpetuating prophecies that are nearly impossible to escape.
its like the famous quote by science fiction writer ARTHUR C. CLARKE, who once noted that "any sufficiently advanced technology is indistinguishable from magic." because of the immense complexity and inherent opacity of these processes within these calculations, we are left to assume that they are upheld to scientific rigor. this assumption is incorrect. these algorithms push forward the prejudices and assumptions of their creators. they are fallible and should be susceptible to outside audit, something that has yet to happen due to legal and political intransigence.
in totality it is a multi-tiered problem that is quite staggering in scale and beyond depressing. given the rapid rate in the innovation and effectiveness of surveillance technologies over the past 40 years, we are only in the infancy of such.
what O'NEIL is calling for is a rethinking of how we use these algorithms. essentially she is advocating for the moral use of them, which seems almost naive given the effectiveness of the monetary benefits to those that employ them. she cites examples by social workers in identifying and addressing specific populations that may be at increased risk needing their services. if used in a benign fashion in order to assist people then there may be hope, but i am skeptical.
the incentives in our society are just too great and our social fabric too weak that i fear this is already an arms race that will define our lives here on out.
this book was next level terrifying, yet written in a sober, methodical fashion that was by no means overwrought or alarmist. its a siren's call like PAUL REVERE in the night warning us all of an impending danger. my fear is we are all asleep, or worse, checking our social media feeds.