Amazon and Microsoft pause police facial recognition and demand regulation

Amazon and Microsoft pause police facial recognition and demand regulation
Image: REUTERS/Mike Segar
We may earn a commission from links on this page.

Within 24 hours of each other, both Amazon and Microsoft this week pledged not to sell facial recognition tools to US police departments, acknowledging the concerns of researchers and activists who say the technology is biased and has grave potential for misuse. The companies’ announcements follow IBM’s decision to disavow facial recognition earlier this week.

In the immediate term, Amazon’s moratorium will have a greater impact than IBM or Microsoft on the use of facial recognition by law enforcement. Amazon sells its product, Rekognition, directly to several law enforcement agencies; Microsoft does not currently sell facial recognition tools to US police departments, and IBM’s market is small. But collectively, the actions of these companies will have a lasting effect on the industry’s development—by ratcheting up pressure on American lawmakers to regulate the technology, for better or for worse.

Amazon’s announcement is more conservative. IBM made a sweeping pledge not to research, develop, or sell facial recognition software to anyone. Amazon, in contrast, announced a one-year moratorium on selling its Rekognition software to police, to give Congress time to “place stronger regulations to govern the ethical use of facial recognition technology.” It will continue to offer the technology to organizations like the International Center for Missing and Exploited Children to investigate cases of human trafficking. The company did not respond when asked if it would continue to research and develop its facial recognition software in the intervening year.

Microsoft, too, called on US legislators to pass federal regulations on the use of the technology without going as far as IBM. Microsoft president Brad Smith said during a Washington Post live event this morning that his company won’t sell facial recognition technology to US police until federal regulations are in place. He warned that Microsoft, Amazon, and IBM can’t act alone. “If all of the responsible companies in the country cede this market to those that aren’t prepared to take a stand, we won’t necessarily serve the national interest or the lives of the Black and African American people of this nation well,” he said, adding that Congressional regulation “is the only way that we will guarantee that we protect the lives of people.”

The announcements follow several years of research, in particular from MIT Media Lab’s Joy Buolamwini and Microsoft Research’s Timnit Gebru, showing that commercial facial recognition tools are much more likely to misidentify darker-skinned women than lighter-skinned men. Amazon’s decision comes after public campaigns from researchers and privacy advocates pressuring the company to accept the researchers’ findings and stop selling Rekognition to law enforcement.

Inioluwa Raji, a tech fellow at NYU’s AI Now Institute who co-authored research with Buolamwini on bias in facial recognition, said the pressure from IBM, Amazon, and Microsoft has big implications for public policy, which will affect the lesser-known firms that do much more business with US police departments. “All three are big companies that are heavily influencing the conversation regulation-wise,” Raji said. If the companies unleash their legions of lobbyists to oppose stringent regulation, then “even a smaller company like Clearview AI can have free rein to do whatever they want.”

But the companies aren’t trying to stop regulation outright; by calling for stronger regulation, they leave open the opportunity to shape new legislation in their favor.

Jameson Spivack, who studies law enforcement’s use of facial recognition at Georgetown University’s Center on Privacy & Technology, said that regulation will ultimately work to the larger firms’ advantage. “The public does not like facial recognition and they’re worried this sentiment will translate into a ban or moratorium, so they want to get out in front of it and coopt it,” Spivack said. “They can support regulation that they can work around, that can give them rules that will allow them to continue to develop the technology.”

Microsoft has a history of lobbying to craft facial recognition regulations at the state level. In Washington, for example, a state senator employed by Microsoft wrote the law that governs facial recognition. California, Maryland, South Dakota, and Idaho have since taken up similar legislation which mirrors the Washington bill word-for-word in some places.

Some researchers worry that this will translate into toothless regulation that allows tech companies to weather the current backlash against police surveillance and then quietly return to business as usual.

“I don’t think Amazon’s statement tells us that there has been a real shift in their position regarding police’s use of facial recognition technology,” said Maria De-Arteaga, an incoming assistant professor at UT Austin who studies ethics in AI. “The way I read their statement, it means that in one year we will be hearing Amazon responding to any critiques by saying that they are following existing regulations.”