AI-Generated Voice Firm Clamps Down After 4chan Makes Celebrity Voices for Abuse
4chan members used ElevenLabs to make deepfake voices of Emma Watson, Joe Rogan, and others saying racist, transphobic, and violent things.
www.vice.com
In one example, a generated voice that sounds like actor Emma Watson reads a section of Mein Kampf. In another, a voice very similar to Ben Sharpio makes racist remarks about Alexandria Ocasio-Cortez. In a third, someone saying "trans rights are human rights" is strangled. In another, Rick Sanchez from the animated show Rick & Morty says "I'm going to beat my wife Morty. I'm going to beat my fucking wife Morty. I'm going to beat her to death Morty." (Justin Roiland, who voices Sanchez, recently appeared in court for a pre-trial hearing on charges of felony domestic violence. Roiland pleaded not guilty in 2020).
This new AI can replicate someone's voice with just a minute of audio, scary stuff that is already being misused... terrible