Weapons of Math Destruction

In 2011, fifth-grade teacher Sarah Wysocki was fired from her job at a Washington, DC public school. Her students, her colleagues, and her community considered her an excellent educator. But a secretive evaluation metric called IMPACT had flagged her as subpar. The algorithm looked at student standardized test scores and concluded Wysocki wasn't measuring up. The problem? Previous teachers in her classroom had cheated on those tests, inflating the baseline so dramatically that no teacher could possibly meet it. Wysocki couldn't appeal. The formula was proprietary, hidden from view. So she left the high-needs DC school for a wealthy Virginia district. The algorithm didn't just fire a good teacher—it took a strong educator away from the students who needed her most.

This is what mathematician Cathy O'Neil calls a Weapon of Math Destruction, or WMD. The term is deliberate. Just like nuclear weapons, these mathematical models operate at enormous scale, cause devastating damage, and their inner workings are hidden from the people they affect most.

Here's how O'Neil defines a WMD. It has three essential components. First, **opacity**—the model's logic is invisible to those being evaluated. You can't see how it works, can't challenge its conclusions, can't correct errors in your data. Second, **scale**—it doesn't judge one person at a time. It processes millions, applying the same flawed logic across entire populations. Third, **damage**—it doesn't just make mistakes. It punishes the poor and marginalized, locking them into cycles of poverty while the wealthy escape scrutiny.

The most dangerous feature of WMDs is that they create what O'Neil calls **toxic feedback loops**. Here's how it works. The model makes a prediction based on flawed data. That prediction shapes decisions about real people. Those decisions produce new data that confirms the original flawed prediction. The model appears accurate—but only because it created the reality it claimed to predict.

In Wysocki's case, the feedback loop worked like this: Cheating by previous teachers created inflated test scores. The IMPACT model used those scores to set impossible standards. Wysocki failed to meet them. She was fired. The model's verdict appeared correct—after all, her students' scores didn't improve. But the model never asked whether the baseline was accurate in the first place.

O'Neil argues that society has accepted a dangerous assumption: that mathematical models are objective. We trust numbers in a way we don't trust human judgment. But every model is built on human choices—what data to include, what to leave out, what to define as success. Those choices encode human prejudice, misunderstanding, and bias. A model's blind spots reflect the priorities of its creators.

Here's the framework for identifying a WMD. Ask three questions about any algorithm that affects your life:

**Is it opaque?** Can you see how the score was calculated? Can you access the data used to judge you? Can you appeal the result? If the answer to any of these is no, you're dealing with a potential WMD.

**Does it operate at scale?** Does this model affect thousands or millions of people? Is it applied uniformly without accounting for individual circumstances? Scale amplifies small errors into massive harm.

**Does it cause damage?** Does the model punish people? Does it deny opportunities, increase costs, or restrict freedom? Does it disproportionately affect the poor, the marginalized, or the vulnerable?

If all three answers are yes, you've identified a Weapon of Math Destruction.

The Wysocki story illustrates all three components. The IMPACT model was opaque—teachers couldn't see how their scores were calculated. It operated at scale—applied to every teacher in the DC school system. And it caused damage—firing good teachers, pushing them out of high-needs schools, and depriving poor students of quality education.

O'Neil makes a crucial point: WMDs don't just make mistakes. They punish the poor while protecting the wealthy. A white-shoe law firm evaluates candidates through personal interviews and recommendations. A cash-strapped urban school district relies on a secret formula. The privileged get human judgment; the masses get machines.

Think about what happened to Sarah Wysocki. She was a teacher in a poor district, evaluated by an opaque algorithm that couldn't account for cheating by previous teachers. She lost her job. Then she moved to a wealthy Virginia school, where presumably she's evaluated by people who can see her teach, talk to her students, and understand her context. The algorithm didn't just cost her a job—it punished the students who needed her most.

This is the hidden threat O'Neil exposes: mathematical models that appear neutral but actually encode human prejudice, operating at massive scale with no accountability. They don't just reflect inequality. They actively create it.

So here's the question that should haunt anyone who cares about fairness: If a machine can fire a beloved teacher and no one can ask why, what happens when the same kind of model decides who gets a loan, who goes to prison, or who gets hired for a job?