You definitely have tried to access some websites and have gotten bombarded with a series of puzzles requiring you to correctly identify traffic lights, buses, or crosswalks to prove that you’re indeed human before you log in.
Known as Completely Automated Public Turing test to tell Computers and Humans Apart (CAPTCHA), the technology is intended to protect a website from fraud and abuse without creating friction. The puzzles are meant to ensure that only valid users are able to access the site and not automated invasions.
Google replaced CAPTCHA with a more advanced tool called reCAPTCHA in 2019, but the team’s technical lead Aaron Malenfant told the Verge at the time that the technology would no longer be viable in 10 years’ time because advanced tech would allow the Turing test to run in the background.
His prediction was right. Artificial Intelligence (AI) bots are fast-evolving and are now beating the reCAPTCHA methodology used to confirm the validity and personhood of the users of various websites. They do this by imitating how the human brain and vision work. In fact, AI bots are measuring up to humans, and even beating them, in numerous facets.
Research shows AI bots beat CAPTCHA and humans
Now, a research paper (pdf) published last month that has yet to be peer reviewed indicates that AI-automated attacks on various CAPTCHA schemes have been successful. The study, conducted by a group of researchers including three from the University of California, Irvine, one from ETH Zurich, one from the Lawrence Livermore National Laboratory, and one from Microsoft, showed that AI bots are now better than humans at decoding the CAPTCHAs. They even create an impression of humans being more robots than the bots that the CAPTCHAs try to keep out. And they even do it much faster.
The researchers recruited 1,400 participants to test websites that used CAPTCHA puzzles, which account for 120 of the world’s 200 most popular websites. “The bots’ accuracy ranges from 85-100%, with the majority above 96%. This substantially exceeds the human accuracy range we observed (50-85%),” the research paper read. “Furthermore the bots’ solving times are significantly lower in all cases, except reCAPTCHA, where human solving time of 18 seconds is nearly similar to the bots’ time of 17.5 seconds.” The study was done on Amazon’s crowdsourcing platform MTurk, and involved various types of CAPTCHAs including identifying chimneys and boats, rotating images, ticking a checkbox, and typing distorted text.
The paper noted that in a contextualized setting human solving time slows down to 22 seconds, indicating that in this more natural setting, AI bots are faster than humans at solving the puzzle.
“We do know for sure that they [the tests] are very much unloved. We didn’t have to do a study to come to that conclusion,” Gene Tsudik, one of the study’s researchers, told the New Scientist. “But people don’t know whether that effort, that colossal global effort that is invested into solving CAPTCHAs every day, every year, every month, whether that effort is actually worthwhile.”
Cengiz Acartürk, a cognition and computer scientist at Jagiellonian University in Kraków, Poland, says that there’s a problem with designing better CAPTCHAs because they have a built-in ceiling. “If it’s too difficult, people give up,” Acartürk says. Whether CAPTCHA puzzles are worth adding to a website may ultimately depend on whether the next step is so important to a user’s experience that a tough puzzle won’t turn away visitors while providing an appropriate level of security.