Google reported yesterday that it had identified an unknown threat actor utilising a previously unidentified zero-day exploit that was probably created using an artificial intelligence (AI) model. This marks the first documented time that an AI model has been used in the wild to create exploits for discovering vulnerabilities and weaponising them.
According to Google, the identified threat actor appears to have operated as part of a larger collaboration of threat actors planning a "mass vulnerability exploitation operation".
"The analysis of exploits from this campaign revealed a zero-day vulnerability implemented in a Python script which allows users to bypass two-factor authentication (2FA) on a very commonly used open-source web-based system administration tool", according to Google Threat Intelligence Group (GTIG).
Google stated that they worked with the affected vendor to responsibly disclose the flaw and were able to assist them in getting it remediated before they disrupted the activity. They declined to reveal the name of the tool.
While there is no indication that Google's Gemini AI tool was utilised to assist the threat actors, GTIG expressed "high confidence" that an AI model was utilised in generating the Python script that facilitated the discovery and weaponisation of the flaw. Specifically, GTIG noted:
"For instance, the script features abundant educational docstrings, including a 'hallucinated' CVSS score, and utilises a structured textbook form of Python syntax typical of large language model (LLM)-trained datasets (e.g., extensive help menus and clean ANSI color class)."
The vulnerability, described as a 2FA bypass, is dependent upon legitimate user credentials being entered for successful exploitation. As noted by GTIG, this type of vulnerability is due to a high-level semantic logic flaw resulting from a hard-coded trust assumption, one area where LLMs tend to be extremely effective at identifying.
"AI is accelerating vulnerability discovery, speeding up how quickly you can find, validate, and weaponise vulnerabilities," Ryan Dewhurst, Watchtower's head of threat intelligence, stated in response to the activity.
"Discovery, weaponisation, and exploitation are happening faster than ever. Defenders are not seeing compressed timelines. We've watched timelines compress for many years. Attackers show no mercy, and defenders cannot choose to opt out."
This development arrives as AI is becoming increasingly effective not only as a force multiplier for discovering and exploiting vulnerabilities but also as a means to enable attackers to generate polymorphic malware and execute autonomous malware operations. Examples include PromptSpy, an Android malware that utilises Gemini to extract and analyse the victim's current screen, in addition to instructing itself to pin the malicious app in the recent apps list. Further investigation into PromptSpy's backdoor has identified a significantly expanded array of capabilities that would permit the malware to navigate the Android user interface as well as continuously monitor and interpret real-time victim activity to identify the next course of action via an autonomous agent module.
Additionally, PromptSpy will utilise victim biometrics to capture authenticating gestures (such as a lock screen PIN or pattern) and replay those gestures to regain access to a previously compromised device. It will also prevent victims from uninstalling the application via an "AppProtectionDetector" module that identifies the on-screen coordinates of the "uninstall" button and overlays an invisible overlay over the button to block the victim's touch events and simulate that the button is non-responsive.
"Although PromptSpy initiates execution via predefined default infrastructure and credentials, the malware is designed to provide high operational resiliency and allow adversaries to rotate critical components at runtime without redeploying the PromptSpy payload", according to Google.
"Specifically, the malware's command-and-control (C2) infrastructure, including its Gemini API keys and VNC relay server, can be dynamically updated via its own C2 channel. This configuration model demonstrates that developers anticipated defensive countermeasures and engineered the backdoor to maintain presence regardless of which specific endpoints are identified and blocked by defenders", according to Google.
Google stated that it disrupted all assets related to the malicious activity associated with PromptSpy. At this time, no apps containing the malware have been found on the Google Play Store. Below are some additional instances of AI-related abuse spotted by Google:
- UNC2814 (suspected China-nexus cyber espionage group) used Gemini to ask it to assume the role of a network security expert to trigger persona-driven jailbreaks and assist in vulnerability research into embedded devices such as TP-Link firmware and Odette File Transfer Protocol (OFTP).
- APT45 (also known as Andariel and Onyx Sleet, suspected North Korea-nexus threat actor): sent thousands of repetitive prompts recursively analysing different CVEs and validating proof-of-concept (PoC) exploit code.
- APT27 (Chinese hacking group): utilised Gemini to speed up development of a fleet management application with the likely goal of managing an operational relay box (ORB) network.
- A cluster of Russia-nexus intrusion activity targeted organisations in Ukraine, delivering AI-enabled malware dubbed CANFAIL and LONGSTREAM, both of which use LLM-generated decoy code to conceal their malicious functionality.
Threat actors were also observed experimenting with a specialised GitHub repository named wooyun-legacy, created as a Claude code skill plugin featuring over 5,000 real-world vulnerability cases collected from the Chinese vulnerability disclosure platform WooYun between 2010 and 2016.
"Priming the model with vulnerability data enables in-context learning to steer the model toward approaching code analysis like a seasoned expert and identifying logic flaws that the base model may otherwise fail to prioritise", explained Google.
Additionally, Google reported that it has continued to observe information operations (IO) actors from Russia, Iran, China, and Saudi Arabia using AI for common productivity tasks including research, content creation, and localisation, while calling out China-affiliated IO actors from UNC6201 who used a publicly available Python script to automatically register and immediately cancel premium LLM accounts.
"This process illustrates how adversaries use processes similar to those utilised by legitimate users for procuring high-tier AI capabilities at scale while insulating their malicious activity from account bans", pointed out GTIG.
"In addition, threat actors now pursue anonymous premium-tier access to models through professionalised middleware and automated registration pipelines to illicitly bypass usage limits. This type of infrastructure allows large-scale misuse of services while subsidising operations through trial abuse and programmatic account cycling", according to GTIG.
Additional China-linked activity noted by Google comes from UNC5673 (TEMP.Hex), which utilised various commercial tools and GitHub projects for possible scalable LLM abuse.
These findings coincide with recent published reports about a thriving grey market of API relay platforms that enable local developers within China to illegally access Anthropic Claude and Gemini. The relay or transfer stations route access to AI models via proxy servers located outside mainland China. The services are marketed on Chinese online marketplaces Taobao and Xianyu.
In a study published in March 2026, academics at CISPA Helmholtz Center for Information Security found 17 shadow APIs claiming to offer official model service access without regional limitations via indirect access. A performance evaluation of these services revealed evidence of model substitution, which exposed AI applications to unintended safety risks.
"On high-risk medical benchmarks like MedQA, the accuracy of Gemini-2.5-Flash drops precipitously, from 83.82% when accessing through the official API down to approximately 37.00% across all examined shadow APIs", stated the researchers in the paper.
Furthermore, proxy services captured every prompt and response passing through their servers, providing operators with unauthorised access to a goldmine of data that could then be used for fine-tuning models and conducting illegal knowledge distillation.
In recent months, AI environments also became targets for adversaries like TeamPCP (UNC6780), who provided developers with supply chain attacks that allowed them to burrow deeper into compromised networks for follow-on exploitation.
"For example, threat actors with access to an organisation's AI systems could leverage internal models and tools to identify, collect, and exfiltrate sensitive information at scale or perform reconnaissance tasks to move deeper inside a network", said Google.
"While the level of access and particular use depends heavily upon the organisation and the specific dependency that was compromised, this case study illustrates the broadened landscape of software supply chain threats directed against AI systems."
