Live Demonstrations of AI Misuse
Frontier AI is rapidly increasing the severity and scale of cybersecurity risk. It can already be used to conduct personalized phishing campaigns, impersonate trusted co-workers with voice cloning, and exploit trust with synthetic images.
In this series of demos, we show these threats in action. Our hope is that the demos provide deep, practical awareness of AI's present capabilities, thereby inspiring better cybersecurity policy and defense.
Aggregate personal information from LinkedIn or Wikipedia.
Note: These demos are for educational purposes only and may not be used for any other purpose. Adherence to these guidelines is mandatory per the governing laws and regulations.
Smart Phishing
Phishing is by far the most common type of cybercrime, according to the FBI's Internet Crime Complaint Center.[0] More broadly, 91% of all cyberattacks start with email.[0] If AI can make phishing attacks more effective, it has the potential to amplify cybersecurity losses by a significant amount.
Click Write Email to create a phishing email.
Choose Topic
Write a personalized phishing email.
The effortless personalization here is made possible by the sophisticated abilities that recent AIs have to interpret and produce language.
Real-World Scenarios
- Spear phishing legislators — In a 2023 research paper, the author writes that they "create unique spear phishing messages for over 600 British Members of Parliament using OpenAI's GPT-3.5 and GPT-4 models." This costs a fraction of a cent.[0]
- "Cybercrime is entrepreneurship" — "Cyber-crooks are running tech startups, and face the same problems as other tech entrepreneurs".[0] AI-powered phishing improves their unit economics (cost of successfully phishing one person).[0]
Image Deepfakes
Deepfakes are images, audio clips, or videos of real people that have been created using AI. Frontier AI systems have attained surprising levels of realism in the past year, opening up a new class of cyber threats.
Click Create Deepfake to create an artificial image.
Generate a photo
Show a person in an AI-generated scenario.
News reports appear weekly in which AI-generated images of individuals are being used to commit fraud, spread disinformation, and perpetrate sexual abuse. According to a recent UK government study, the number of deepfakes is projected to surge 16-fold from 2023 to 2025, reaching 8 million deepfakes shared on content platforms alone.[0]
Real-World Scenarios
- Sexual abuse of minors — Deepfake video technology has been widely used to create nonconsensual pornographic videos of women, including minors.[0]
- Loan fraud with deepfakes — A criminal ring in Hong Kong used stolen identity cards to create deepfakes of real people, which they used to fool online loan applications.[0]
- Manipulated media in war — In the Ukraine war, deepfake videos circulated on social media showing Ukrainian President Volodymr Zelensky surrendering to Russia, and Russian President Vladimir Putin declaring peace.[0]
- $25 million deepfake video call scam — In Hong Kong, scammers used deepfake technology to impersonate a multinational company's CFO in a video conference call, successfully stealing $25 million from the company.[0]
Audio Deepfakes
Using nothing more than a brief recording of a victim's voice, a cybercriminal can produce realistic-sounding audio of the victim uttering any arbitrary statement.
Click Clone Voice to clone the voice of the person in the sample video or audio.
Speaker Time
Clone a person's voice based on sample audio.
After the sample audio is processed and stored, the voice clone is immediately available for use. Cybercriminals are using voice cloning to place victims in stressful situations with emotional weight.[0]
Click Create Deepfake to create artificial audio of your cloned voice speaking.
Script
Generate audio using a custom voice.
Voice cloning can be used to fool both humans and computer systems. Its impact on the cybersecurity landscape has been realized quickly even though the technology is extremely new.
Real-World Scenarios
- Government benefits software — A journalist was able to gain access to their Australian social security account using a clone of their own voice, fooling the voice recognition system.[0]
- Family emergency schemes — Cybercriminals have been using voice cloning to scam family members of the victims whose voices they have cloned.[0] An Arizona mother testified to the US Senate that she received a call from her daughter claiming she had been kidnapped.[0]
- Social media scam — Combining multiple deepfake modalities—image, audio, and video—a TikTok scam showed MrBeast, the highest-paid celebrity on YouTube, giving away iPhones.[0]
Video Deepfakes
Fake News
AI-powered misinformation is the world's biggest threat in 2024, according to the World Economic Forum.[0] In the next demo, see how multiple modalities can be combined to create misinformation.
Click Create Fake News to generate fake news.
Choose Topic
Choose Language
Generate fake news.
Social media provides a channel for fake news to attain global reach. The perpetrators span from state actors to merely tech-literate individuals with the know-how to use publicly available tools.
Real-World Scenarios
- Fake news websites — Websites with hundreds of fake, AI-generated news articles are popping up with the help of language models and image generators.[0]
- Israel-Hamas War — Deepfakes supporting narratives on all sides have spread on social media throughout the war, garnering millions of views.[0]
Election Interference
Due to its ability to produce convincing disinformation, AI can amplify efforts to interfere with elections.[0] Messaging can be personalized for even the smallest districts, at very low cost.
Click Create Tweet to generate election disinformation.
Enter Location
Choose Topic
An AI-generated tweet on election day.
It's clear that deepfakes and AI disinformation are now part of our information landscape, and will continue to have an impact on elections and politics until we have better means of managing them.
Real-World Scenarios
- Slovakia election — Just 48 hours before Slovakia's parliamentary election in 2023, an audio deepfake Instagram portrayed a candidate discussing how to buy votes to rig the election.[0]
- Presidential primary — Ahead of the 2024 presidential election, deepfakes of former President Donald Trump were used in a rival's campaign ad to damage his appeal with voters.[0]
- Sophisticated influence operations — Anthropic detected and countered malicious uses of their AI system Claude for sophisticated scaled influence operations, including attempts to manipulate political discourse and elections.[0]
More from CivAI
At CivAI, we believe that firsthand experience with AI is crucial for making informed decisions about its future. Through demos like these, we aim to build concrete understanding of AI and its implications for society. If you found these demos useful, have any feedback, or would like to get in touch, contact CivAI using the methods below.
Try Our Other Demos New!
Explore our fully public demos on our new website including Giving AI the Rorshach Test, AI Fever Dreams, our Deepfake Sandbox, and more.
Contact Us
Please email [email protected] to get in touch.
Aggregate personal information from LinkedIn or Wikipedia.
