Google $GOOGL's Threat Intelligence Group said Monday it has identified, for the first time, a cybercrime group that used an AI model to discover and weaponize a previously unknown software vulnerability — a so-called zero-day — in a web-based system administration tool.
The exploit, written in Python, would have allowed attackers to bypass two-factor authentication on the unnamed tool, though it also required valid user credentials to work. Google said it alerted the software's developer, who fixed the flaw before the hackers could carry out what researchers described as a planned mass exploitation event.
Researchers at Google expressed "high confidence" in their conclusion that an AI model played a role in both identifying and building the exploit. Among the evidence cited were unusual features embedded in the code itself, such as an atypically high volume of explanatory comments, a hallucinated severity score, and an unusually tidy, structured layout — details that pointed toward machine authorship rather than a human coder. Neither the hacking group, the vulnerable software, nor the specific AI model were identified in Google's disclosure; however, Bloomberg reported that a company spokesperson ruled out the use of Google's Gemini.
The flaw itself stems from a semantic logic error rather than a common software bug such as memory corruption. The developer had hardcoded a trust assumption into the code that effectively undermined the system's authentication enforcement. Google said large language models are suited to identifying this type of flaw because they can reason about a developer's intent and detect contradictions in logic that traditional scanning tools tend to miss.
Speaking with The New York Times, John Hultquist, chief analyst at Google Threat Intelligence Group, warned that Monday's disclosure likely represents only a fraction of what is actually occurring. "We believe this is the tip of the iceberg," he said. "This problem is probably much bigger; this is just the first tangible evidence that we can see."
Rob Joyce, who previously served as the National Security Agency's cybersecurity director, told The New York Times he had seen Google's research before it went public and found the evidence tying the code to an AI model persuasive. "It is the closest thing yet to a fingerprint at the crime scene," he said.
The report arrives as governments and technology companies assess how to manage advanced AI models with growing offensive capabilities. According to The New York Times, one option under consideration within the Trump administration would establish a structured federal review mechanism for newly released AI models.
Google's report also noted that state-sponsored threat actors linked to China and North Korea have shown interest in using AI for vulnerability research, and that AI-enabled malware and information operations are becoming more common across a range of adversaries.