Top Ten Malicious Uses of GenAI

AI Spotlight
The Different Faces of Malicious GenAI Usage

The nerds at Google DeepMind analyzed 200 media reports covering nefarious usage of GenAI from January 2023 to March 2024.

Their goal was simple. Classify the current threats and tactics to help guide future decisions on combatting the threat of GenAI misuse. As students of cybersecurity and fans of how deepfakes are changing the social engineering game, we can benefit from their learnings to understand how threats are evolving.

Let’s start with the punchline. They found that most reported misuse cases were super simple. Said more directly, using existing technologies, even your grandmother can create a deepfake and start a hardened life of cybercrime. Can you blame her? Inflation is killing retirement budgets. Don’t forget to tell her about not moving to Florida…

Their research found the following three themes and ten tactics:

Realistic Depictions of Human Likeness

• Impersonation: Assume the identity of a real person and take actions on their behalf (aka deepfakes)

• Appropriated Likeness: Use or alter a person’s likeness or other identifying features.

• Sockpuppeting: Create synthetic online personas or accounts. This is popular in misinformation campaigns.

And the more disturbing ones…

• Non-consensual Intimate Imagery (NCII): Create sexual explicit material using an adult person’s likeness. This is a growing problem.

• Child Sexual Abuse Material (CSAM): Create child sexual explicit material. This is just disgusting and I hate that we even have to talk about this but it’s a growing problem in schools and needs to be addressed.

Realistic Depictions of Non-Humans

• Falsification: Fabricate or falsely represent evidence (misinformation)

• Intellectual Property (IP) Infringement: Use a person’s IP without their permission

• Counterfeit: Reproduce or imitate an original work, brand, or style and pass as real

Use of Generated Content

• Scaling and Amplification: Automate, amplify, or scale workflows

• Targeting and Personalization: Refine outputs to target individuals with tailored attacks

They also analyzed the modality (aka the type of media) of the created media. As you can see below, deepfake audio was the #1 culprit for impersonation attacks. That’s unsurprising, given that voice-cloning software is eerily good at creating near-exact replicas of anyone’s voice. And the capabilities to create deepfake videos are not far behind with open-source software capable of swapping out your face.

The immediate concerns from this research show that disinformation and social engineering have leveled up. While that may not seem surprising, it’s helpful for organizations to see where threats are going and how to focus their budget on combating new forms of social engineering. And for society, it’s more reason to slow down and verify your sources.

Security Deep Dive
Infostealers Are So Hot Right Now

There’s a phrase in security that “hackers don’t hack in, they log in.” And one way they do that is with stolen credentials. In some cases, those credentials can be exposed in other data breaches. For users that reuse passwords (no one here…right?…), attackers can snag those and log into different sites with the same credentials.

And then there’s the more active way of stealing credentials. And infostealers are all the rage now. If you haven’t heard of them, they’re malware that steal sensitive information, including your passwords.

Just check out this advertisement from the Atomic MacOS Stealer (AMOS) creator, which I learned about from an article g0njxa wrote.

This ad tells you everything you need to know about the data they steal. Some of which might be surprising. It includes:

1. System Info: This includes your password and your keychain file. Many users think their passwords are safe using Mac’s built-in keychain capabilities. Bad news…with your password, which the infostealer collects, it can unlock your keychain file and grab your password…ouch.

2. Browsers: It supports every major browser and steals all the things you don’t want to be stolen: stored passwords, credit cards, wallets, auto-fills, and cookies. The cookies are extra concerning because all those sites that you stay logged into happen because of a cookie stored in the browser. When the attacker gets those, they can take over your session…even MFA can’t stop that.

3. Crypto wallets: If you’re a crypto nerd, those browser extensions for your wallet are an infostealer’s best friend.

Infostealers are one of the biggest threats faced by the average person and company today. For the average user, all their accounts could be at risk if they’re not using a password manager that isn’t impacted by infostealers. For companies, one set of credentials stolen either from the company laptop or even an employee’s personal device can result in the compromise of their network or data.

Security News
What Else is Happening?

💰️ Crypto scammers created deepfakes of Prince William and UK Prime Minister Keir Starmer to encourage people to put money into the “National Invest Platform”, a fake investment scheme. Check out both videos here.

👋 Law enforcement seized the infrastructure of the Radar/Dispossessor ransomware group. The group has only been around for a year but managed to attack 43 companies across the US, South America, India, Europe, and UAE.

🧪 A Texas-based chemical manufacturer you’ve never heard of but cuts million-dollar checks reported they lost $60M after attackers socially engineered an employee into wiring money to bank accounts they control.

🇮🇷 OpenAI banned accounts linked to Iranian influence operations that used ChatGPT to generate content. This included generating long-form articles on US politics and comments for social media posts.

🚪 An attacker broke into Crypto company Unicoin’s Google G-Suite account and changed passwords for all its users, effectively locking them out. They didn’t recover access for four days. They’re still assessing what the attackers did with their access.