As Facebook CEO Mark Zuckerberg faced America’s top politicians this week, they struck again and again at the question of harmful, malicious, or misleading content on Facebook, and asked: Does Facebook have a plan?
Zuckerberg responded that one is in the works—the company was hiring 20,000 people around the world to moderate the content generated by more than 2 billion Facebook users. And long-term, that job would be handled by artificial intelligence at massive scale.
“We’re going to shift increasingly to a method where more of this content is flagged up front by A.I. tools that we develop,” he said. “We’ve prioritized the most important types of content that we can build A.I. tools for today, like terror-related content.”
Zuckerberg says that 99% of the terrorism-related content that Facebook takes down is taken down before it’s reported. That’s a good metric—but Zuckerberg admits this content is some of the easiest to find.
What’s more difficult is finding hate speech, bullying, and threats made on Facebook—subjects that rely heavily on context, are often implied, and can be interpreted in multiple ways.
The best example of this lives on the internet today: Google and Jigsaw, two Alphabet companies, have a tool called Perspective that’s meant to help identify whether text online is “toxic.” But the system confuses words with negative connotations with negative or harmful speech, as Quartz has previously reported.
Through this online hate-speech detector, “garbage truck” was found to be 78% toxic, while “race war now” was only 24% toxic. This happens due to the way that the most popular form of artificial intelligence today, deep learning, is trained to learn. Put simply, an engineer shows the deep learning system many examples of good things and bad things. By pulling these good things apart, like the individual words in a non-harmful sentence against those in one deemed harmful, it learns the relationships between those words and constructs a complex web of good and bad.
But the AI system can only learn from data that humans have marked good or bad in a machine-understandable way. At the time of Perspective’s release, Jigsaw said that the AI system’s shortcomings were due to a lack of this labelled data. It hadn’t seen enough examples of these complex, content-heavy examples to learn that race war is bad in some contexts, but not in a scholarly article or history book.
Facebook’s challenge is finding all these permutations of hate speech, bullying, threats, and terrorism, in order to train its AI to look out for similar examples. The problem gets messier because not everyone can agree what makes a post harmful or abusive.
Facebook faced this problem in 2016 when the social media platform was deluged by reports of fake news during the US presidential election. While Zuckerberg cited AI as a potential savior to the problem then, the company now is hiring 20,000 humans to oversee all content moderation.
Mevan Babakar, head of automated fact checking at UK non-profit Full Fact, says that regardless of whether AI or humans are used to moderate content, both methods share the common question about who determines whether certain speech is acceptable.
“AI comes with big questions on definitions. Hate speech is sometimes obvious but in some cases people can’t agree. The same is true for matching factchecks to content. Who will be making these choices? Small choices have big consequences here,” Babakar said.
Facebook has more control over choosing humans in its hiring process of moderators to make those decisions rather than an algorithm making uninterpretable decisions. While Facebook might be able to get rid of the most obvious examples easily, there will always likely be tough, situational decisions that humans will have to make.
“Mark Zuckerberg is being realistic when he says ‘we will have A.I. tools that can get into some of the nuances’ in 5-10 years, but there are a lot of nuances and not all of them will be able to be automated away,” Babakar says.
Other researchers disagree on Zuckerberg’s timeline, with the disclaimer that technological predictions are always difficult to make.
Delip Rao, co-founder of the Fake News Challenge and CEO of R7 Speech Sciences, says that focusing on AI as the only avenue for battle hate speech could lead to bind spots in thinking. Regulation, education, platform design, and AI tools to assist moderators find content could all be alternatives to an AI-first approach.
“From everything we’re seeing in state-of-the-art in [natural language processing] research, I highly doubt the seriousness of the 5-10 year timeline to combat hate speech online using AI/automated approaches,” Rao tells Quartz over direct message. “We can be hopeful while continuing to work under the assumption that it is unrealistic.”