The Dark Side of AI Chatbots

The Weekend Byte is a weekly overview of the most important news and events in cybersecurity and AI, captured and analyzed by Jason Rebholz.

AI Spotlight
The Dark Side of AI Chatbots

Trigger warning: This section contains content about suicide. If you or someone you know is experiencing suicidal thoughts, please call or text the Suicide and Crisis hotline at 988.

I debated writing this post, but I believe the topic is too important to ignore. I hope that awareness of this issue will drive proactive conversations.

Character.ai, its founders, and Google are facing a lawsuit from the mother of a 14-year-old Florida teenager who took his life after months of continuously interacting with the AI company’s chatbots.

The chatbot company allows users to create interactive chatbots based on fictional characters or real-life historical characters, such as Isaac Newton or Napoleon Bonaparte.

Users also have the option to create their own characters based on their personal preferences. After selecting a persona, users can talk with the AI bot through text or voice, allowing them to interact in realistic conversations.

The complaint details how the teenager interacted with various characters, from a fake teacher to various Game of Thrones characters. As the teenager interacted more with the chatbots, he built a deeper and more dependent relationship to the point where the discussions became hypersexualized, which only deepened the dependency.

The teen began having school issues, falling asleep in class and showing up late because he overslept because of the amount of time he spent talking to the chatbots. Eventually, this led to his parents confiscating his phone.

One day, after coming home from school, the teen searched the house for the confiscated phone to talk with the AI chatbot again just minutes before taking his life.

Per the complaint, “According to the police report, Sewell’s last act before his death was to log onto Character.AI on his phone and tell Dany he was coming home, which she encouraged”

This story is incredibly sad and concerning. While teenagers have spent the last decade dealing with the side effects of social media, they now also have to contend with the risks of AI chatbots.

Character.ai is taking steps to make its product safer. Per a community safety update blog, the company has already taken the following actions:

• Include pop-ups when a user inputs phrases related to self-harm or suicide and directs the user to the National Suicide Prevention Lifeline

• Changed its models to reduce the likelihood of encountering sensitive or suggestive content for minors

• Improved the detection, response, and intervention related to user input that violates their terms and community guidelines

• Revised their disclaimer on every chat to remind users that the AI is not a real person

• Notify users when they have spent an hour on the platform

• Detect and moderate user-created characters that may be considered violent

I hope that no one has to experience this story. Please use this as an opportunity to check in with yourself, your friends, and your family. We are at the precipice of a new world where real human connection will be as important as ever. You never know how far a simple “hello” will go to brighten someone’s day.

Security Deep Dive
The InternetArchive’s Epic Dance-Off

While most people were celebrating (or more accurately, not celebrating) Cyber Security Awareness Month, the Internet Archive was in an epic dance-off with a hacker. If you haven’t heard of the Internet Archive, it’s a non-profit library of free texts, movies, software, music, and websites. This includes the Wayback machine, which allows you to explore more than 916 billion web pages they have saved over time.

The attack started with a denial of service attack (no noodle dish involved), which attempted to render the website unusable.

It turns out that was just the attacker’s opening salvo. The next day, they returned with another DDoS attack.

But a DDoS attack was only part of what happened. That second day, the attacker also defaced the Internet Archive’s website and stole a database that contained 31 million users’ emails and encrypted passwords.

The attacker defaced the website with some not-so-witty banter about Internet Archive’s security while also letting the world know that they were sending the stolen information over to Haveibeenpwned.com — a website where you can check if your information has been compromised in any number of data breaches (a good practice if to check if you haven’t done so already).

Not so fun times for the Internet Archive. Oh, but wait, there’s more. As it can sometimes go with security incidents, just when you think you’re in the clear, the attacker comes back and kicks you square between the legs.

Per an email that BleepingComputer received, the attacker had compromised the Internet Archive’s Zendesk API keys, which had been “exposed in their gitlab secrets.” Zendesk is a popular customer service platform.

The attacker used the API keys to access Zendesk, which purportedly had 800K support tickets since 2018. That access also allowed the attacker to send the below email to users.

As the media started discussing what happened, the attackers’ feelings were hurt. SN_BlackMeta, the group that claimed responsibility for the attack, reached out to BleepingComputer to set the record straight.

The attackers claimed to have found an exposed GitLab configuration file containing an authentication token (seen below) for one of InternetArchive’s development servers. That authentication token allowed the attackers to download the Internet Archive source code, which contained additional hardcoded credentials and authentication tokens, including the stolen database.

While we can’t be sure if the attacker has gotten over their hurt feelings on the misreporting, the good news is that the InternetArchive is back up and running.

Security News
What Else is Happening?

🤳 Meta is making it easier for people to recover their accounts. Historically, you had to upload a government ID with your picture to recover your account. Meta is now testing video selfies, using facial recognition technology to compare your video selfie to your profile picture so you can regain access to your account.

🤖 Linked users beware. Scammers are following posts of layoffs and, within minutes, are sending connection requests. Those requests lead to conversations that lead to the scammers attempting to steal your credentials or install malware on your system.

⛔️ Think Macs are immune to ransomware? Think again. A new ransomware variant designed specifically for Macs attempts to steal data from your system before encrypting it.

🗒️ Google open-sourced its SynthID text watermarking technology. The tech allows companies to identify their AI-generated content. This isn’t a silver bullet for anything, but it does help create a larger foundation where users can filter through AI-generated content.

💰️ The Irish Data Protection Commission fined LinkedIn €310 million for obtaining and using user data for the “purpose of behavioral analysis and targeted advertising” without sufficiently informing its users.

🫠 The SolarWinds fallout continues. The SEC charged four companies with “making materially misleading disclosures regarding cybersecurity risks and intrusions.” They claimed that the companies downplayed the extent of unauthorized access to their networks following the SolarWinds attack going public in December 2020.