It’s likely not news to you that women experience harassment on Twitter. But a new study from researchers at Amnesty International and artificial intelligence software provider Element AI attempts to shed some more light on just how big this problem is. And it’s bad out there.
To perform their analysis, researchers collected tweets directed at 778 female journalists and politicians in the US and UK in 2017. Because spotting abuse can be tricky, they enlisted volunteers, which they called the “Troll Patrol,” to assess whether a sample of 288,000 tweets contained what could be considered abusive or problematic content.
From that sample data, an algorithm developed by Element AI estimated that 7.1% of the total 14.5 million tweets directed at the women contained abusive or problematic language. Because the researchers did not have access to any tweets that may have been removed by Twitter for abuse, that estimate could be a conservative one.
Women of color were targeted at a disproportionately high rate: compared to white women, they were 34% more likely to be the target of hateful tweets. Black women faced the most abuse, with 84% more likely to be targeted than white women. “We could see that abuse—sexist, racist, homophobic abuse—targeted women’s identities,” says Milena Marin, a coordinating researcher of the study. “The more intersectional identities you have, the nastier the abuse.”
Marin and her colleagues at Amnesty International have implored Twitter to be more transparent about abuse on their platform, from how many tweets are reported, and how human moderators and algorithms make decisions about reported tweets, to what actions Twitter takes in response to reported tweets. “They are the gatekeepers of the information,” says Marin. “But the most basic questions, like how many moderators you have, they can’t answer.”
The researchers also want Twitter to make anonymized datasets available for researchers to analyze. Organizations are allowed to collect tweets, and many do using third party apps, but the timescale and scope of data available for download is limited. Twitter’s terms and conditions also prohibits researchers from sharing collected data widely. “We shouldn’t have to engage a team of people around the world to help us understand the scale of this problem,” Marin says. “But they don’t want to provide information, so it’s important for us to keep up the pressure.”
When contacted for comment, Twitter referred Quartz to a formal response sent by Twitter general counsel Vijaya Gadde to Amnesty International on Dec. 12 in response to their research findings. “Twitter’s health is measured by how we help encourage more healthy debate, conversations, and critical thinking,” Gadde wrote. “Conversely, abuse, malicious automation, and manipulation detract from the health of Twitter. We are committed to holding ourselves publicly accountable towards progress in this regard.”
Even if Twitter were completely transparent, it’s far from clear how to better manage harassment on social media. The first problem is catching it in the first place; but there are only so many human moderators available, and algorithms just aren’t yet good enough at understanding important context to flag problematic tweets satisfactorily. “If an algorithm flags the word ‘kill,’ there’s a big difference between ‘I will kill you’ and ‘I would kill for a beer,'” says Marin. An algorithm, too, might miss cues a human would easily pick up on, like adding asterisks in place of letters in a slur (e.g., “b*tch”).
According to Gadde, Twitter does take context into account when evaluating reported tweets and deciding what action to take, like “whether the behavior is targeted at an individual or group of people” and “whether the behavior is newsworthy and in the legitimate public interest.”
Even if hateful tweets are identified, it’s a subjective call whether a tweet is abusive or problematic. In Gadde’s reply to Amnesty International, she noted that what defines a “problematic” tweet and whether it should be removed from the site “warrants further discussion.” Twitter has struggled in the past with how to address harassment on its platform, and has faced wide criticism for its lack of action.
Marin says Amnesty International “included problematic tweets because it is important to highlight the breadth and depth of toxicity on Twitter in its various forms and to recognize the cumulative effect that problematic content may have on the ability of women to freely express themselves on the platform,” Marin says. A single user tweeting “get back to the kitchen” at a woman may be seen as offensive but not abusive enough to warrant an immediate account ban. But ask any woman on Twitter: If you get just a few of those not-quite-abusive tweets a day, the effect is something that certainly feels like abuse.