Google: How Threat Actors Use Gemini for Theft & Espionage

When Gen AI and agentic AI first appeared on the scene, the world seemed full of possibilities. However, those possibilities are not uniformly positive. 

As the AI Threat Tracker report from the Google Threat Intelligence Group (GTIG) reveals, cybercriminals are increasingly turning to AI for assistance in their malicious campaigns. 

From state-sponsored threat actors using AI for espionage to private companies enlisting AI for intellectual property theft, Google’s threat report paints a detailed picture of the adversarial use of AI.

What are distillation attacks?

As AI becomes a part of the business ecosystem, companies are investing large sums of money for specialised training of their AI. This is especially the case when the companies offerings are AI-powered services. 

But through a new form of cyber attack, known as a "distillation attack", cybercriminals are able to steal the knowledge of a proprietary AI system, thereby infringing the intellectual property of the owner of the LLM.

In the past, bad actors had to physically or digitally break in to steal companies’ trade secrets. Now, they can simply distil out the information from their LLMs. 

This is done by taking advantage of a legitimate ML training technique called knowledge distillation (KD) where new models learn from bigger, teacher models.

Threat actors use input-output pairs from the LLMs of proprietary AI models to cheaply train their own “student models”.

These attacks (also known as "model extraction attacks") are able to mimic the performance of the “teacher model”, creating a cheap replica of the original model.

Google DeepMind and GTIG say that these attacks are on the rise and Google’s Gemini is in the crosshairs as threat actors try to coerce the model to reveal its complete reasoning only for it to be stolen away. 

“Google’s latest AI Threat Tracker marks a specific turning point: we are no longer just worried about bad prompts, but the industrial-scale extraction of the models themselves,” says Jamie Collier, Lead Advisor (Europe) at Google Threat Intelligence Group on LinkedIn. 

“Model extraction attacks occur when an adversary uses legitimate access to probe a mature machine learning model, with the information extracted used to train a new model. 

“We haven’t seen APTs do this, but we’ve blocked many model extraction attempts from private companies all over the world.”

Nation state threat actors using Gemini 

The GTIG report revealed that state sponsored APT actors are increasingly using Gemini in all phases of their attack cycles all the way from reconnaissance to data exfiltration. 

Iranian Government backed cyber threat actor APT-42 was discovered using Gen AI models like Gemini to conduct background research on officials, so as to create convincing fake personas which high ranking people are likely to engage with for targeted social engineering. 

The North Korean state-backed actor UNC2970, linked to the Lazarus group, had targeted major cybersecurity and defence companies by posing as recruiters to create “high-fidelity phishing personas”, which would help in their spear phishing and whaling tactics. 

Google disabled all the assets associated with these threat actors upon discovery.

Agentic AI for malware   

To make attacks even easier, cybercriminals are increasing automating attacks using agentic AI. 

A Chinese state-sponsored cybercrime group, APT 31 – also known as Zirconium, Violet Typhoon, Judgment Panda and Altaire – GTIG has discovered were using the Hexstrike MCP tooling to analyse vulnerabilities of certain US targets.

The cybercriminal group achieved this by pretending to be security researchers pen testing websites. 

Other cybercrime groups linked to China, UNC795 and APT41 were also found to have used Gemini to troubleshoot code which they used in their intrusion activities. 

Gemini was leveraged to develop and deploy malicious tooling, and as the report sites, they also used Gemini for “AI-integrated code auditing capability, likely demonstrating an interest in agentic AI utilities to support their intrusion activity”.

Google identified a plethora of AI enabled tools developed specifically for the purpose of assisting cybercriminals in various darknet forums and marketplaces. These tools offered free services which were plagued with advertisements and paid ad-free versions reflecting the pricing system of commercial products. 

These incredibly complex tool and services shows that there is a vast, financially and politically motivated ecosystem that is behind AI assisted cybercrime and it looks like they are here to stay.