In2003, the philosopher Nick Bostrom proposed a thought experiment on the dangers of artificial intelligence. In his scenario, an AI was tasked with maximizing the number of paper clips in its collection. A superintelligent machine unshackled from human perspectives and ethical frameworks, he argued, would deduce that the best way to maximize its paper-clips collection was to convert all matter in the universe—humans included—into paper clips. The question of how to create the ultimate paper-clips collection would be definitively answered, but at what cost?
In his new book The Chaos Machine: The Inside Story of How Social Media Rewired Our Minds and Our World, the journalist Max Fisher sets out to analyze a similar situation. In his scenario, libertarian-minded Silicon Valley entrepreneurs, in an attempt to maximize online engagement, created esoteric self-evolving algorithms that convert all social-media users into pure incandescent and polarized outrage, and then shrug at the wreckage they have caused. Fisher’s work could have served as a vital warning sign against this mathematically precise descent into madness, but the book he has written suggests instead that the eponymous Chaos Machine is already so alarmingly efficient that it even managed to coopt Fisher’s thoughts before he set them to paper. Those thoughts prove to resemble those of the supposedly manipulated, blinkered online radical communities he highlights.
The Chaos Machine examines a serious problem that very clearly transcends race, gender, class, and politics through a completely polarized lens. In Fisher’s view, algorithm-driven extremism on the right is an approaching landslide of evil brought on by knowing bad actors and the tacit acceptance of their more mainstream counterparts. But he ignores and downplays any equivalent phenomenon on the left, aside from the occasional sad head shake at how unsuspecting but well meaning people are brainwashed by the white male priests of the Techno-Anarchist Church of Zuckerberg into engaging in “so-called cancel culture.”
It cannot be stressed enough that the crisis Fisher is describing is real and pressing. In their obsessive pursuit of greater engagement, social-media companies (Facebook and YouTube being the primary focus of the book) have designed self-iterating “Deep Learning” algorithms that inexorably funnel users toward increasingly radicalized content, since those algorithms have learned how that kind of mat-
erial brings about more active engagement.
Fisher describes in detail how algorithm-driven polarization in countries as various as Myanmar, Sri Lanka, India, Germany, Brazil, and the United States fuels xenophobia, conspiracies, and violence—problems that concern the companies reaping the benefits from the increasingly universal usage of social media only once it’s too late to fix what’s broken. Fisher establishes the foundations of the problem—from the dominant traits of Silicon Valley whiz kids, to the evolution of news feeds and engagement features, to the psychological and sociological effects of the rapid transition to ubiquitous online life. But he himself is so trapped in the ideological bubble curated for him by the very Chaos Machine he criticizes that he can identify it as a problem only when it afflicts those who live outside his social biodome.
Take, for instance, Gamergate—a portmanteau referring to an incredibly complex and confusing 2014–15 Internet controversy long since forgotten. Fisher mentions Gamergate more than 60 times, including in the name of a chapter that boldly declares “Everything Is Gamergate.” It was a toxic period of culture-war battles within the gaming community, and he believes it is perhaps the central hotbed of all Internet foulness over the past eight or so years. Fisher’s Gamergate is the primordial ooze that spawned online racist and misogynist harassment, neo-Nazism, Pizzagate, mass-shooting incels, Trump’s election, Qanon, and, ultimately, the insurrection on January 6, 2021.
In the interests of brevity, I won’t relitigate Gamergate here, but Fisher’s description of it and the “experts” he cites come entirely from the version of events fed to him by the Chaos Machine. The algorithms that dominated Fisher’s feed at the time presented him with one-sided evidence that it was purely a right-wing harassment campaign against women and minorities. The truth is far more complicated and nuanced, though no less toxic—perhaps even more so, because of the toxicity on the part of the left that he either has no idea about or ignores conveniently.
Gamergate was merely one niche flare-up in the increasing polarization Fisher describes. But it is hard to reconcile the seriousness of his concern with the paucity of his effort to disentangle it from his own political biases. For example, Fisher frequently mentions the clinical psychologist and author Jordan Peterson as being “one of the online alt right’s most important gateways.” He portrays Peterson’s content as adjacent to beliefs about “white genocide and Jewish subjugation,” because some users who watch his videos on YouTube are eventually led by the recommendation algorithm to videos on those evil topics. But if that’s the case, then logic requires one to point out that Peterson isn’t extreme—for otherwise he would be the destination, not the starting point. If every rabbit hole begins at the surface, how can one blame the surface for the depth of the hole?
The writer and researcher Renée Diresta, whose work on the online radicalization of the anti-vaccine movement is heavily featured here, first became aware of the problem because the benign parenting groups she had joined on Facebook eventually led her to anti-vaxxer content. By Fisher’s logic, Facebook groups on sleep training or teething are therefore complicit as ideological gateways to radicalization—even though their connection to more extreme content is forced algorithmically and not a result of ideological affinity. “It was the algorithm that drove them,” admits Fisher in a moment of clarity, explaining in a later section on medical misinformation that “as with political videos, the algorithm used more credible or familiar channels as gateways to direct users toward the worst conspiracies and misinformation.”
The book features considerable shoe-leather reporting and robust scholarly work in the fields of psychology, social sciences, and data analysis. But the studies Fisher cites are almost always framed as windows into right-wing extremism. He discusses one in which researchers presented a false headline to Republicans about 500 migrants from Central America who had been arrested wearing suicide vests. According to the study, many Republican participants said they would share the headline on social media despite thinking it was inaccurate, indicating it was because the story reflected their politics. While it is true that the study itself (and subsequent media write-ups about it) used the question Fisher mentions as a representative example, it was not the only question researchers had posed, and the study was not exclusively conducted on Republicans. The authors observed similar behavior in both ideological camps, stating in their conclusions that “the pattern of sharing intentions we observe here matches the pattern of actual sharing observed in a large-scale analysis of Twitter users, where partisanship was found to be a much stronger predictor of sharing than veracity.”
Diresta also frequently mentioned the pan-partisan nature of the anti-vax community, especially prior to the Covid-19 pandemic, telling PBS in 2018 that “the anti-vaccine movement is actually bipartisan or nonpartisan, depending on how you want to frame it.”
Speaking of vaccines, Fisher widely features anti-vax attitudes as an example of a radicalization rabbit hole driven by social media. He points to the spread of conspiracies related to the Zika vaccine in Brazil in 2016. What he doesn’t bother to mention is that these conspiracies were not driven by right-wing attitudes but rather over the false claim that pesticides, rather than the Zika virus, were causing an outbreak of microcephaly—a claim echoed on social media by the progressive actor Mark (the Hulk) Ruffalo. Rather than use this example to highlight the pan-partisan nature of online conspiracy theories, Fisher instead points to the case of a Brazilian woman named Gisleangela Oliveira dos Santos who was taken in by anti-Zika vaccine conspiracies related to microcephaly on YouTube and later voted for right-wing Brazilian presidential candidate Jair Bolsonaro.
Now, it is true that some of theextreme examples of right-wing radicalization that Fisher cites don’t currently have equivalently dangerous counterparts on the left. But he specifically mentions that years of neglecting to focus on these radicalizing phenomena helped them breed unmolested. His one-sided focus in The Chaos Machine risks doing just that for his side of the aisle. Members of Congress such as Marjorie Taylor Greene and Lauren Boebert are called “fringe conspiracists” who succeed only by plucking extremist strings online, but there is no mention of congresswomen Rashida Tlaib or Ilhan Omar, who have both shared anti-Semitic blood libels and engaged in racist dog-whistling on social media to the tune of tens of thousands of shares and likes.
Indeed, every time Fisher has an opportunity to be an honest broker about the effect these algorithms have on both sides, he shows himself to be a lost cause. Given a chance to mention the violence surrounding Black Lives Matter protests in the summer of 2020—which was fueled by hyper-polarized social-media content, video snippets of police brutality of varying authenticity and context, and self-interested radical political activists with huge digital pulpits—Fisher instead chooses to insinuate that at least some of the violence was the result of false-flag-style operations by the supposedly right-wing Boogaloo Bois in an attempt to spark bloodshed between authorities and protestors. He zeroes in, for instance, on one incident in May 2020, when a man named Ivan Harrison Hunter fired a rifle into the air, hoping “to spark violence between protestors and police that would escalate into war,” though no one was hurt in the incident.
Given a chance to mention the reams of wrong and deceptive information circulating during Covid, not just by Donald Trump, but also against him, Fisher chooses to repeat the claim that Trump, from the White House podium, said “the virus could be cured by drinking diluted bleach”—a claim that one quick Google search would have shown him is simply not true. Instead of highlighting how the algorithm corrupts us all, Fisher instead highlights how it has already corrupted him. He is just a paper clip in its collection.
Photo: Today Testing
We want to hear your thoughts about this article. Click here to send a letter to the editor.
