Voice Cloning and Audio Deepfakes

How Voice Cloning Works

Modern voice cloning requires as little as three seconds of audio to create a synthetic replica of someone's voice. Services like ElevenLabs, Resemble.AI, and open-source tools like VALL-E and Tortoise TTS analyze the unique characteristics of a voice — pitch, cadence, accent, breathing patterns — and build a model that can speak any text in that voice.

The technology has legitimate uses: dubbing films into other languages while preserving the original actor's voice, helping people who have lost their voices due to illness, and creating audiobook narration. But the same technology enables fraud, political manipulation, and harassment at a scale that was previously impossible.

In January 2024, voters in New Hampshire received robocalls featuring a synthetic version of President Biden's voice telling them not to vote in the primary. The calls were convincing enough that the FCC issued an emergency declaratory ruling making AI-generated voice calls illegal under existing robocall laws.

Voice Cloning in the Wild

The scale of voice-based fraud has exploded. In 2023, the FTC reported that AI voice scams cost Americans an estimated $2.7 billion, with criminals cloning family members' voices from social media clips to make emergency calls demanding money. A typical scam involves cloning a grandchild's voice from a TikTok video and calling the grandparent claiming to be in jail or a hospital.

In politics, audio deepfakes pose a distinct threat because they are harder to debunk than images. A fake audio recording of a politician making racist remarks or accepting a bribe can spread through messaging apps like WhatsApp and Telegram where there is no content moderation. In Slovakia's 2023 election, a fabricated audio recording of a liberal candidate discussing plans to raise beer prices and rig the election was shared widely in the final 48 hours before voting, when fact-checkers had little time to respond.