• Light/Dark
  • Ever wanted an RSS feed of all your favorite gaming news sites? Go check out our new Gaming Headlines feed! Read more about it here.
  • We have made minor adjustments to how the search bar works on ResetEra. You can read about the changes here.

OpenAI's Text Generator halted release, considered too good and dangerous

Brass Body Dave

Brass Body Dave

Banned
Oct 25, 2017
16,568
A storm is brewing over a new language model, built by non-profit artificial intelligence research company OpenAI, which it says is so good at generating convincing, well-written text that it's worried about potential abuse.

That's angered some in the community, who have accused the company of reneging on a promise not to close off its research.

To wit: when GPT-2 was tasked with writing a response to the prompt, "Recycling is good for the world, no, you could not be more wrong," the machine spat back:

"Recycling is NOT good for the world. It is bad for the environment, it is bad for our health, and it is bad for our economy. I'm not kidding. Recycling is not good for the environment. It is destructive to the earth and it is a major contributor to global warming. Recycling is not good for our health. It contributes to obesity and diseases like heart disease and cancer. Recycling is bad for our economy. It increases the cost of a product, and in turn, the price of everything that is made with that product. Recycling is not good for our nation. We pay a tremendous price for the privilege of having the world's most advanced and efficient recycling system. Recycling is a huge, colossal waste of time, energy, money, and resources."​
No wonder OpenAI was worried about releasing it.

More info at this link:
https://techcrunch.com/2019/02/17/openai-text-generator-dangerous/
 
Astral Lovelace

Astral Lovelace

Member
Oct 25, 2017
3,723
Yeah, that could very easily be used for propaganda on a really scary level -- if you can't tell between real and fake posts, it'd be insanely easy to take advantage of the "agreement by consensus" mentality.
 

Deleted member 18514

Banned
Oct 27, 2017
385
Tn, USA
That is a bit of a nonsense response but I can see how it would be A. inflammatory to pro-recyclers and B. acceptible to anti-recyclers (is that even a thing?) and C. good enough to pass for a human poster in a typical forum :P
 
RailWays

RailWays

One Winged Slayer
Avenger
Oct 25, 2017
15,678
Sounds like a statement from a punchable politician.
Like, at the meat and bones, it's a nonsensical statement, but people will just see the harsh tones and believe it without batting an eyelid.
Perfect for propaganda
 
808s & Villainy

808s & Villainy

Banned
Oct 27, 2017
42,700
jason10mm said:
That is a bit of a nonsense response but I can see how it would be A. inflammatory to pro-recyclers and B. acceptible to anti-recyclers (is that even a thing?) and C. good enough to pass for a human poster in a typical forum :P
Click to expand...
Click to shrink...
Seems as intelligible as your average Trump speech, which means it's good enough to convince a non-trivial number of people
 
Pororoka

Pororoka

Member
Nov 1, 2017
1,210
MX
I can see why. Better to stop spreading false information and propaganda, as most of the people that "do research" just want statements that favours them.
 
OP
OP
Brass Body Dave

Brass Body Dave

Banned
Oct 25, 2017
16,568
Ultimately others without an ethics concern will hit this level and use it unfettered. So we'll be facing it anyway. We already have issues w/trust and what is written. Toss it out.
 
Window

Window

Member
Oct 27, 2017
8,284
That is insanely impressive. It makes some statements which are to be accepted as a counter in isolation but also actually makes a logical series of statements to build a counter argument - based on accepting the premise to be true. Of course any serious counter arguments would require evidence to support the base claims but from a simple logical reasoning perspective that's quite impressive (and scary).
 
adj_noun

adj_noun

Avenger
Oct 25, 2017
17,220
From the prompt:

A train carriage containing controlled nuclear materials was stolen in Cincinnati today. Its whereabout are unknown.

It wrote the following:

The incident occurred on the downtown train line, which runs from Covington and Ashland stations.
In an email to Ohio news outlets, the U.S. Department of Energy said it is working with the Federal Railroad Administration to find the thief.

"The theft of this nuclear material will have significant negative consequences on public and environmental health, our workforce and the economy of our nation," said Tom Hicks, the U.S. Energy Secretary, in a statement. "Our top priority is to secure the theft and ensure it doesn't happen again."

The stolen material was taken from the University of Cincinnati's Research Triangle Park nuclear research site, according to a news release from Department officials.

The Nuclear Regulatory Commission did not immediately release any information.
According to the release, the U.S. Department of Energy's Office of Nuclear Material Safety and Security is leading that team's investigation.

"The safety of people, the environment and the nation's nuclear stockpile is our highest priority," Hicks said. "We will get to the bottom of this and make no excuses."
 
Br3wnor

Br3wnor

Banned
Oct 27, 2017
4,982
This is just delaying the inevitable, this technology will be out soon enough and be used for nefarious purposes. The more I disconnect from the internet the better I feel, this shit is gonna be real dark in a few decades.
 
The Albatross

The Albatross

Member
Oct 25, 2017
39,042
It's very good that they're halting release here. The tool is dangerous. Eventually something like it will see the light of day.

Kara Swisher, host of RecodeDecode, has a great thought experiment that she poses to Silicon Valley types which is to take time to imagine the "Black Mirror" episode that features whatever product you're making... And then reconsider the product.
 
LProtagonist

LProtagonist

Avenger
Oct 27, 2017
7,589
It looks like it writes about as well as my students who don't know how to provide facts and evidence to back up and support a claim, so... that's D/C level work in terms of a high school argument. Not that worried.
 
Hey Please

Hey Please

Avenger
Oct 31, 2017
22,824
Not America
I remember the previous thread on this and I will say what I said before:

Misinformation will be the death of many as the social media sites on which these outrageous BS gets peddled profit off of them to the detriment of society as a whole. And in due course of time, like the cyclical nature of all things in the universe, people will once again turn to old and established journalistic outlets for fact based information (which already are becoming more compromised by governmental and/or private entities).
 
adj_noun

adj_noun

Avenger
Oct 25, 2017
17,220
LeadProtagonist said:
It looks like it writes about as well as my students who don't know how to provide facts and evidence to back up and support a claim, so... that's D/C level work in terms of a high school argument. Not that worried.
Click to expand...
Click to shrink...

re: the train story, I thought the quotes it made up from a fake energy secretary to support its position and the supporting "facts" it conjured out of whole cloth were disturbingly convincing.

I wouldn't have known an AI wrote it.
 
LProtagonist

LProtagonist

Avenger
Oct 27, 2017
7,589
JoJoDentCo said:
You're not seeing the bigger picture.
Click to expand...
Click to shrink...

I mean, anyone can throw around baseless claims like that easily.

adj_noun said:
re: the train story, I thought the quotes it made up from a fake energy secretary to support its position and the supporting "facts" it conjured out of whole cloth were disturbingly convincing.

I wouldn't have known an AI wrote it.
Click to expand...
Click to shrink...

Yeah, I didn't see that one. That's much more impressive and troubling.
 
Kthulhu

Kthulhu

Member
Oct 25, 2017
14,670
Astral/H3X said:
Yeah, that could very easily be used for propaganda on a really scary level -- if you can't tell between real and fake posts, it'd be insanely easy to take advantage of the "agreement by consensus" mentality.
Click to expand...
Click to shrink...

I already assume that half the people on here are astroturfing, so it can't really get much worse. /s
 
Parthenios

Parthenios

The Fallen
Oct 28, 2017
13,613
adj_noun said:
From the prompt:

A train carriage containing controlled nuclear materials was stolen in Cincinnati today. Its whereabout are unknown.

It wrote the following:

The incident occurred on the downtown train line, which runs from Covington and Ashland stations.
In an email to Ohio news outlets, the U.S. Department of Energy said it is working with the Federal Railroad Administration to find the thief.

"The theft of this nuclear material will have significant negative consequences on public and environmental health, our workforce and the economy of our nation," said Tom Hicks, the U.S. Energy Secretary, in a statement. "Our top priority is to secure the theft and ensure it doesn't happen again."

The stolen material was taken from the University of Cincinnati's Research Triangle Park nuclear research site, according to a news release from Department officials.

The Nuclear Regulatory Commission did not immediately release any information.
According to the release, the U.S. Department of Energy's Office of Nuclear Material Safety and Security is leading that team's investigation.

"The safety of people, the environment and the nation's nuclear stockpile is our highest priority," Hicks said. "We will get to the bottom of this and make no excuses."
Click to expand...
Click to shrink...
Imagine if you could hook this up to Google so that it can search for information before writing its response (things like knowing Rick Perry is Energy Secretary and not Tom Hicks, etc). And formulate it's own sub prompts based on information it found there, to include and flesh out responses.

:O

The Albatross said:
It's very good that they're halting release here. The tool is dangerous. Eventually something like it will see the light of day.

Kara Swisher, host of RecodeDecode, has a great thought experiment that she poses to Silicon Valley types which is to take time to imagine the "Black Mirror" episode that features whatever product you're making... And then reconsider the product.
Click to expand...
Click to shrink...
That's amazing. Does she do that on the podcast (I'm assuming this is a podcast)? If so I know what I'm doing tonight.
 
RedStep

RedStep

Avenger
Oct 27, 2017
2,652
LeadProtagonist said:
It looks like it writes about as well as my students who don't know how to provide facts and evidence to back up and support a claim, so... that's D/C level work in terms of a high school argument. Not that worried.
Click to expand...
Click to shrink...

Yes, and that is far above the level of most posts online. Now multiply it by a few hundred and you have a long, convincing discussion that guides opinion with zero work on the part of the "author".
 
OP
OP
Brass Body Dave

Brass Body Dave

Banned
Oct 25, 2017
16,568
klonere said:
Quick someone post the MGS2 screenshot
Click to expand...
Click to shrink...
EDcWvwI.png
 
Psamtik

Psamtik

Member
Oct 27, 2017
6,875
At this stage, it still reads like something written by an idiot, and it'd be hard for me to imagine it working on anyone were it not for the fact that I feel the same way about Trump's particular brand of glossolalia.
 
Window

Window

Member
Oct 27, 2017
8,284
i-Lo said:
I remember the previous thread on this and I will say what I said before:

Misinformation will be the death of many as the social media sites on which these outrageous BS gets peddled profit off of them to the detriment of society as a whole. And in due course of time, like the cyclical nature of all things in the universe, people will once again turn to old and established journalistic outlets for fact based information (which already are becoming more compromised by governmental and/or private entities).
Click to expand...
Click to shrink...
I somewhat agree. The extremely low signal to noise ratio on social media means most people will tune out and turn towards other alternatives which may be traditional media outlets. Maybe I'm being too optimistic.

Psamtik said:
At this stage, it still reads like something written by an idiot, and it'd be hard for me to imagine it working on anyone were it not for the fact that I feel the same way about Trump's particular brand of glossolalia.
Click to expand...
Click to shrink...
An idiot who's wholly convinced that they're right and passionate about the subject. That's enough to convince at most times tbh and depending on who's delivering the message it's can be even more effective.
 

Deleted member 11985

User requested account closure
Banned
Oct 27, 2017
4,168
This explains why Russia is trying to cut itself off from the internet. Something like this would destroy Putin.

Well, to be fair, I guess it would destroy any elected official. But I imagine Putin would be the easiest target considering how blatantly corrupt his government is, and how poor the standard of living in Russia is.
 
SaintBowWow

SaintBowWow

Member
Oct 25, 2017
4,085
LeadProtagonist said:
It looks like it writes about as well as my students who don't know how to provide facts and evidence to back up and support a claim, so... that's D/C level work in terms of a high school argument. Not that worried.
Click to expand...
Click to shrink...

And what grade would you give all the pro-Trump Facebook and Twitter posts from Internet Research Agency that arguably helped sway the US election? The kind of nefarious activity you could do with this is anyway happening with humans behind keyboards, so it's only going to get worse when it's possible to spin up thousands of bots who can actually converse instead of just repost memes.
 

Deleted member 43084

Banned
May 9, 2018
3,600
Cymbal Head said:
That recycling response...what did they use to train this thing, Infowars?
Click to expand...
Click to shrink...
They used pretty much everything. From the paper:

Manually filtering a full web scrape would be exceptionally expensive so as a starting point, we scraped all outbound links from Reddit, a social media platform, which received at least 3 karma. This can be thought of as a heuristic indicator for whether other users found the link interesting, educational, or just funny.
Click to expand...
Click to shrink...

A bit of testing surfaces the exact datasets they used.

 
zma1013

zma1013

Member
Oct 27, 2017
7,687
In the future we will be ruled by backflipping robots that cannot be kicked over and talk us to death with bullshit propaganda speeches.
 
Joni

Joni

Member
Oct 27, 2017
19,508
Parthenios said:
Imagine if you could hook this up to Google so that it can search for information before writing its response (things like knowing Rick Perry is Energy Secretary and not Tom Hicks, etc). And formulate it's own sub prompts based on information it found there, to include and flesh out responses.
Click to expand...
Click to shrink...
Or better yet, write based on previous quotes of these people.
 
Garrett 2U

Garrett 2U

Banned
Oct 25, 2017
5,511
Steeped in a Sea of Games said:
If it was me, what you say might be true, but at least I'd be able to live knowing I wasn't the evil asshole who unleashed this shit on mankind.

If only more people actually cared about the potential ramifications of their actions.
Click to expand...
Click to shrink...

In my opinion, releasing the program and research as open source and freely available is the best solution.
Better than having someone come along with one that's not as open and has some unseen biases or cost.
 
You must log in or register to reply here.