• Light/Dark
  • Ever wanted an RSS feed of all your favorite gaming news sites? Go check out our new Gaming Headlines feed! Read more about it here.
  • We have made minor adjustments to how the search bar works on ResetEra. You can read about the changes here.

AI-Generated Voice Firm Clamps Down After 4chan Makes Celebrity Voices for Abuse (NSFW)

AImalexia

AImalexia

Prophet of Truth
Member
Aug 31, 2021
2,426
www.vice.com

AI-Generated Voice Firm Clamps Down After 4chan Makes Celebrity Voices for Abuse

4chan members used ElevenLabs to make deepfake voices of Emma Watson, Joe Rogan, and others saying racist, transphobic, and violent things.
www.vice.com www.vice.com

In one example, a generated voice that sounds like actor Emma Watson reads a section of Mein Kampf. In another, a voice very similar to Ben Sharpio makes racist remarks about Alexandria Ocasio-Cortez. In a third, someone saying "trans rights are human rights" is strangled. In another, Rick Sanchez from the animated show Rick & Morty says "I'm going to beat my wife Morty. I'm going to beat my fucking wife Morty. I'm going to beat her to death Morty." (Justin Roiland, who voices Sanchez, recently appeared in court for a pre-trial hearing on charges of felony domestic violence. Roiland pleaded not guilty in 2020).
Click to expand...
Click to shrink...

This new AI can replicate someone's voice with just a minute of audio, scary stuff that is already being misused... terrible
 
ezekial45

ezekial45

Member
Oct 25, 2017
6,752
I firmly believe this free for all we're seeing with AI right now is going to come to a head into spectacular fashion, which still hasn't happened yet.
 
CupOfDoom

CupOfDoom

Member
Dec 17, 2017
3,141
How naive do you have to be to not immediately see this coming.

I swear that if tech people in general spent more than half a second thinking of the ethical implications of the tech they want to make, the world would be a much better place.
 
Temascos

Temascos

Member
Oct 27, 2017
12,521
This is so utterly unsurprising, how do the people who make this stuff not realise their tech can be misused? I guess they actually do know and it's part of some weird plan with a funder who is like one of those religious fanatics trying to bring about the destruction of the world and rip society apart.
 
PlanetSmasher

PlanetSmasher

The Abominable Showman
Member
Oct 25, 2017
115,742
CupOfDoom said:
How naive do you have to be to not immediately see this coming.

I swear that if tech people in general spent more than half a second thinking of the ethical implications of the tech they want to make, the world would be a much better place.
Click to expand...
Click to shrink...

Tech bros: "but I CAN do it, so doesn't that mean that I SHOULD?"
 
Nepenthe

Nepenthe

When the music hits, you feel no pain.
Administrator
Oct 25, 2017
20,699
CupOfDoom said:
I swear that if tech people in general spent more than half a second thinking of the ethical implications of the tech they want to make, the world would be a much better place.
Click to expand...
Click to shrink...
Science: We're all about could've, not should've.

To be fair to the STEM kids, a lot of what we're seeing with AI now is the result of unrelated, concurrent study coming to a head and clashing. Some of these developments have been unpredictable.
 
TheBrightestSun

TheBrightestSun

Banned
Jul 22, 2022
1,867
We will have AI revolution within the next couple of this. AI that is able to replicate voices, AI that is able to generate text and respond, AI that is able to create fakes...

I wonder how many people will use that to record the voices of people they are missing or something and then using stuff like ChatGPT to generate conversation with a people who are already dead.
 
Zhengi

Zhengi

Avenger
Oct 28, 2017
1,901
Your Scientists Were So Preoccupied With Whether Or Not They Could, They Didn't Stop To Think If They Should.jpg
 
NickatNite

NickatNite

Member
Oct 27, 2017
5,257
California
Adobe introduce/talked about a similar software ~3/4 years ago.

Could there be actual good use for tech like this, Yes. But, there will obviously be bad case scenarios as well.
 
Z-Beat

Z-Beat

One Winged Slayer
Member
Oct 25, 2017
31,849
ezekial45 said:
I firmly believe this free for all we're seeing with AI right now is going to come to a head into spectacular fashion, which still hasn't happened yet.
Click to expand...
Click to shrink...
When someone deepfakes the entirety of Congress and uploads a "recorded" Congressional assembly to the internet
 
boontobias

boontobias

Avenger
Apr 14, 2018
9,539
If I was making a AI Voice Mimic app, off the top of my head the top 3 ways people would use it are:

1. Replacing voice actor wages
2. Online trolls
3. tiktoks

Are the first two really worth the last one
 

SasaBassa

Member
Oct 25, 2017
7,078
Everyone who could rub two brain cells together immediately knew this and parallel technology would be a pox.

Need the descrambler software alongside far more verification than ever before— and fast. No doubt there will be counters. However, the damage this stuff will do in the interim is crazy.

People can't even use Google right. We're so boned with fake shit because we won't be able to share reality. Buckle in for the stupid adventures of Lois and Clark.
 
m_shortpants

m_shortpants

Member
Oct 25, 2017
11,246
CupOfDoom said:
How naive do you have to be to not immediately see this coming.

I swear that if tech people in general spent more than half a second thinking of the ethical implications of the tech they want to make, the world would be a much better place.
Click to expand...
Click to shrink...

I agree, but the genies been out of the bottle for a while. It's better to have this technology out in the open, so it can be studied, hopefully regulated, and we can find ways to detect when something is Ai generated.

The cynic in me would rather it like this in the open rather than in the hands of governments and private entities.
 

Deleted member 23094

Banned
Oct 28, 2017
2,217
I love how far computer generated voices have come. It won't be long before the are completely indistinguishable from real humans. Then after that comes simultaneous translation into another language while maintaining the original speakers' voice characteristics, making it possible to communicate naturally and in real time without needing interpreters. It's sad to me that so many people only ever focus on the negative aspects of what new technology allows for.
 
The Albatross

The Albatross

Member
Oct 25, 2017
39,038
It's a product that would exist seemingly only for abuse. It's hard to think of anything useful for this that isn't exploitative.

Ponpoko said:
I love how far computer generated voices have come. It won't be long before the are completely indistinguishable from real humans. Then after that comes simultaneous translation into another language while maintaining the original speakers voice characteristics, making it possible to communicate naturally and in real time without needing interpreters. It's sad to me that so many people only ever focus on the negative aspects of what new technology allows for.
Click to expand...
Click to shrink...

Good counter finding one useful/not-harmful aspect of this.
 
Katbobo

Katbobo

Member
May 3, 2022
5,392
If something like this becomes possible in real time, i think there's really cool uses for it. Roleplaying games and being able to switch into a character voice, or trans people being able to pass online easier, or just being able to choose a voice the same way we choose an avatar online or other ways in which we present ourselves.

But it's also terrifying it can just copy people's voices 1:1 like that. It seems like they have some feature coming to eventually generate a "new" voice for someone. It seems like limiting it to that would be the best way to enable the positive options while not letting someone make a celeb read racial slurs.
 
ClearMetal

ClearMetal

Member
Oct 25, 2017
15,314
the Netherlands
All this AI technology feels like the 21st century version of Marie Curie et al. experimenting with radiation and other nuclear shit in their shed, except on a much wider scale.

Maybe in 50 years we will look back on the current internet wild west and shake our heads. "Can you believe they just threw that into our laps and let us run wild?"
 
KtotheRoc

KtotheRoc

One Winged Slayer
Member
Oct 27, 2017
56,662
This technology is going to be used against politicians. Imagine the kind of language that people will hear someone say.

This technology could start wars.

I don't have a lot of hope for the human race.
 
lorddarkflare

lorddarkflare

Member
Oct 25, 2017
6,258
Nepenthe said:
Science: We're all about could've, not should've.

To be fair to the STEM kids, a lot of what we're seeing with AI now is the result of unrelated, concurrent study coming to a head and clashing. Some of these developments have been unpredictable.
Click to expand...
Click to shrink...

More importantly, while the STEM kids may be somewhat ignorant of the potential uses for their tech, they are not the decision-makers. The business folks are the ones steering the ship, and asking them to make this stuff. Any time unethical tech comes into being the question should be "who paid to make it" not "who made it".
 
Nepenthe

Nepenthe

When the music hits, you feel no pain.
Administrator
Oct 25, 2017
20,699
Z-Beat said:
When someone deepfakes the entirety of Congress and uploads a "recorded" Congressional assembly to the internet
Click to expand...
Click to shrink...
I suggest doing an Eli Lilly where they're faked making the world a better place, like they pass universal healthcare or some shit.
 
OP
OP
AImalexia

AImalexia

Prophet of Truth
Member
Aug 31, 2021
2,426
combine this voice tech with this

www.resetera.com

Ars Technica: AI image tech can now create life-wrecking deepfakes with ease

It's already happening to Youtubers by the droves. One Youtuber I follow was forced to make a video saying how there are dozens of pornos with her in it which have all been AI generated, including her voice... This is scary because until we have a way to punish people who ruin the image of...
www.resetera.com www.resetera.com

and anyone can be anyone over video :^)
 
Teiresias

Teiresias

Member
Oct 27, 2017
8,223
Anyone using an AI to clone someone's voice they don't explicitly have a release to do should be held criminally liable.
 
mbpm

mbpm

Member
Oct 25, 2017
23,618
AImalexia said:
combine this voice tech with this

www.resetera.com

Ars Technica: AI image tech can now create life-wrecking deepfakes with ease

It's already happening to Youtubers by the droves. One Youtuber I follow was forced to make a video saying how there are dozens of pornos with her in it which have all been AI generated, including her voice... This is scary because until we have a way to punish people who ruin the image of...
www.resetera.com www.resetera.com

and anyone can be anyone over video :^)
Click to expand...
Click to shrink...
And this story goes hand in hand with that one streamer recently who got caught on stream with deepfaked porn videos which involved some of his female streamer friends in one of his tabs.
 
Vapelord

Vapelord

Member
Oct 27, 2017
1,830
Montreal
Meanwhile, there are tons of Twitch streamers that have these AI celebrity voices that spit out random TTS messages from donors all day every day...
 
Jordan117

Jordan117

Member
Oct 27, 2017
1,998
Alabammy
The tech really is incredible. It not only sounds real, with audible breaths and a natural rhythm, but it has the spooky ability to reflect the emotional context of the words, sounding bright and chipper for ad copy, slow and heavy for dramatic passages, even reading dialogue like "X whispered" or "Y said sarcastically" in the appropriate tone. And its ability to mimic a voice within just a few seconds is like magic.

I'm glad they're watermarking the end result but they really need to publish a detector and go after those abusing it. Probably also need some stronger identity verification like DALL-E to enforce IP bans. This has so many positive applications, but if they're putting this out into the world for free use then they need to take responsibility for the harm that's done with it, too.
 
Pellaidh

Pellaidh

Member
Oct 26, 2017
3,177
lorddarkflare said:
More importantly, while the STEM kids may be somewhat ignorant of the potential uses for their tech, they are not the decision-makers. The business folks are the ones steering the ship, and asking them to make this stuff. Any time unethical tech comes into being the question should be "who paid to make it" not "who made it".
Click to expand...
Click to shrink...

Yup. The people doing this aren't scientists, they're businessmen (the public LinkedIn profiles of the people in charge of this company make this abundantly clear).

Sadly, we live in a world where doing actual science is grossly underpaid and underappreciated, while unethical garbage like this makes millions if not billions of dollars. And this difference between science and industry is particularly pronounced in the tech field.
 
mbpm

mbpm

Member
Oct 25, 2017
23,618
Jordan117 said:
.

I'm glad they're watermarking the end result but they really need to publish a detector and go after those abusing it. Probably also need some stronger identity verification like DALL-E to enforce IP bans. This has so many positive applications, but if they're putting this out into the world for free use then they need to take responsibility for the harm that's done with it, too.
Click to expand...
Click to shrink...
And that's without people questioning the veracity of the detectors too
 
TheBrightestSun

TheBrightestSun

Banned
Jul 22, 2022
1,867
KtotheRoc said:
This technology could start wars.
Click to expand...
Click to shrink...
And we are now moving towards AI controlled drones and vehicles. Wars will literally turn into a games for a highly developed societies.


The problem with this is not being used for fake speech, but it being used for not believing in truth. To tackle that we need a really serious critical thinking, but oh well..
 
Last edited:
Nepenthe

Nepenthe

When the music hits, you feel no pain.
Administrator
Oct 25, 2017
20,699
TheBrightestSun said:
The problem with this is not being used for fake speech, but it being used for not believing in truth. For tackle that we need a really serious critical thinking, but oh well..
Click to expand...
Click to shrink...
You can be a great critical thinker and still be susceptible to falling for AI generated content. That's what happens we become capable of essentially replicating reality. We're going to need a whole new set of skills to develop in discerning what is and isn't bullshit, and good luck with that when we've got a third of the country believing in Q Anon.
 

Zamorro

Member
Oct 25, 2017
269
If this AI technology becomes prevalent I can see people going from "believing everything on the internet" right into "believing nothing is real on the internet".

Maybe in the future I will just assume that 90% of you are just fake AI posters.
 
Moebius

Moebius

Member
Oct 28, 2017
5,395
Most of these companies do not let you clone a voice that isn't your own. Strange that they let you do that.

bangai-o said:
Are actor likeness protected by law? A.I. could just make the next Taylor Swift album.
Click to expand...
Click to shrink...

AI created music is going to be REALLY interesting. Feed an AI the entire Metallica discography and see what it can come up with. I imagine in the future, there will be great songs created by AI that mimic the style of the band they were trained on.
 
You must log in or register to reply here.