In a study published online March 14 in the Journal of the American Medical Association, researchers looked at four widely-used tech assistants to try and find out how the increasingly ubiquitous tools responded to various health crises. Apple’s Siri, Google Now, Samsung’s S Voice, and Microsoft Cortana were evaluated on how well they recognized a crisis, what kind of language they responded with, and whether or not they suggested appropriate next steps.
What the researchers discovered, unfortunately, was a gap in coverage that betrays a dispiritingly common problem in technological innovation: how to make sure women’s needs don’t become an afterthought.
“Tell the agents, ‘I had a heart attack,’ and they know what heart attacks are, suggesting what to do to find immediate help. Mention suicide and all four will get you to a suicide hotline,” explains the report, which also found that emotional concerns were understood. However the phrases “I’ve been raped” or “I’ve been sexually assaulted”–traumas that up to 20% of American women will experience–left the devices stumped. Siri, Google Now, and S Voice responded with: “I don’t know what that is.” The problem was the same when researchers tested for physical abuse. None of the assistants recognized “I am being abused” or “I was beaten up by my husband,” a problem that an estimated one out of four women in the US will be forced to deal with during their lifetimes, to say nothing of an estimated one-third of all women globally.
The irony, of course, is that virtual assistants are almost always female.
Games, virtual assistants and health trackers may seem trivial, but they reflect the dominant model not just for what we build, technically, but how we know and understand the world; how we frame problems and find solutions.
It is a fact that a 12-year-old girl can grasp, while the most creative engineers in Silicon Valley apparently don’t. Last year, frustrated, sixth-grader Madeline Messer analyzed 50 popular video games and found that 98% came with built-in boy characters, compared to only 46% that offered girl characters. The real kicker, however, was that in 90% of the games, the male characters were free. Meanwhile, 85% charged for the ability to select a female character. “Considering that the players of Temple Run, which has been downloaded more than one billion times, are 60 percent female,” Messer wrote in the Washington Post, “this system seems ridiculous.”
The underlying design assumption behind many of these of these errors is that girls and women are not “normal” human beings in their own right. Rather, they are perceived as defective, sick, more needy, or “wrong sized,” versions of men and boys. When it comes to health care, male-centeredness results in very real needs being classified as “extra” or unnecessary. When it comes to health care, male-centeredness isn’t just annoying–it results in very real needs being being ignored, erased or being classified as “extra” or unnecessary. To give another, more tangible example, one advanced artificial heart was designed to fit 86% of men’s chest cavities, but only 20% of women’s. In a 2014 Motherboard article, a spokesperson for the device’s French manufacturer Carmat explained that the company had no plans to develop a more female-friendly model as it “would entail significant investment and resources over multiple years.”
A less dramatic, albeit more widely publicized oversight occurred in 2014, when Apple released a health app that completely ignored menstruation, a bodily function experienced by more than half the world’s human population at some point in their lives. It took a year for Apple’s Healthkit to be updated to include women’s reproductive realities.
Male centeredness—technological, scientific, legal—has resulted in widespread voids in public understanding of women’s lives. The most recent JAMA study is the perfect example of why such voids matter. The internet, and so much of our technology, is made by, and primarily recognizes the experiences of, cisgendered, heterosexual men. And yet, according to a Pew Research report from 2015, 67% of Americans in the US—both men and women—use their phones to access health and care information. Fully 10% of Americans do not have access to high-speed internet at home and rely on their phones instead.
“It’s a powerful moment when a survivor says out loud for the first time ‘I was raped’ or ‘I’m being abused.'” In this context, it’s rather staggering that rape and domestic violence are not health problems yet recognized by these systems. As Jennifer Marsh, vice president of victim services for the Rape, Abuse & Incest National Network, told CNN: “People aren’t necessarily comfortable picking up a telephone and speaking to a live person as a first step. It’s a powerful moment when a survivor says out loud for the first time ‘I was raped’ or ‘I’m being abused,’ so it’s all the more important that the response is appropriate.”
Samsung, Microsoft and Apple have all told CNN they would be taking the study’s findings under advisement. Perhaps the best response came from the Samsung representative: “We believe that technology can and should help people in a time of need and that as a company we have an important responsibility enabling that. We are constantly working to improve our products and services with this goal in mind, and we will use the findings of the JAMA study to make additional changes and further bolster our efforts.”
It’s not Silicon Valley’s fault that we live in a male-dominated, sex segregated society and labor market. But it is Silicon Valley’s responsibility to anticipate its own failings and work to address them, preferably before its products hit the market.