Microsoft AI shutdown for being racist

[vanished]

1 minute ago, Roawoao said:

The AI was most certainly not put in a blank slate into the world it was trained on many countless internal databases and tuned by hand to make it appear as it did initially. The problem was that their moderation system and update system was allowing trolls to enter whatever they wanted into the training data and eventually that swamped out any of the original data resulting in the Nazi/Hitler tilt. There were built in resistance to more recent politically sensitive events but they didn't understand how good trolls are at finding your software bugs.

Ok maybe I worded that wrong, but my idea was that with enough "pre-training", the stuff it learns on twitter would only ever constitute a smaller portion, and that would help a lot, no?

[Roawoao]

12 minutes ago, Ryan_Vickers said:

Ok maybe I worded that wrong, but my idea was that with enough "pre-training", the stuff it learns on twitter would only ever constitute a smaller portion, and that would help a lot, no?

Well if they designed the original pre-training to be immutable (super enforced) and always overpowering the new data it would not appear to be very natural if you asked it the right questions. The key thing is to let people enter whatever they want but then with multiple TAYs so to speak there would be differences between them at the end of the day. This coupled with manual/automated filtering of inputs you can prune out the garbage and update the AI when it "sleeps". So that the next day all the troll stuff gets filtered out by hand or automated filters and a breed/synchronized TAYs that appear to be working properly are used in the next day. If you rinse and repeat you should be able to have a fairly stable chatbot system that takes into account non-troll words. IP banning, account shadowbans, the abuse TAY instance, ... would all greatly help some deeper integration with the social platform would allow you to run a council of TAYs with one or two dedicated to responding to spam/trolls getting reset periodically and only training the anti-spam/troll detection system but never synchronizing with the production chat data. (A honey pot basically)

 

One other thing to do is to program in detection for attempts to store specific messages with the chatbot as this could become a form of secret group communication or novel abuse (say a new current event being trolled). Big data analysis on the new data every day would help detect such hotspots or question patterns leading to lengthy responses.

 

[LAwLz]

9 minutes ago, Roawoao said:

No it wouldn't be fairly miserable it would be terribly inaccurate as it demonstrated. Trivia questions, asking its opinion about subjects no one has previously talked to it about, asking it to make original thought, ... all not going to work since it is still a DNN training set driven chatbot. Watson is free for anyone to use and it isn't exactly helping IBM out much. 

What are you on about? I am starting to suspect that you are a DNN chat bot because you throw out random facts even when they are not related to the conversation. What does Watson being free for anyone to use have to do with anything?

Anyway, Watson can give input on topics nobody has ever talked to it about before. It's not like someone had been inputting every possible subject and answer to it before the match of Jeopardy. As for giving its opinion on subjects, well duhh we just got to the stage where a super computer can interpret natural language. Of course it won't be able to perfectly replicate human behavior yet. But in the future it might very well be able to.

You are moving the goalpost by the way. You specifically said: "if you tried asking it say computer questions with easy google fu answers it would likely fail miserably". You said it would fail at trivia questions (questions which can easily be answered by a Google search) and now you changed the subject to opinions on subjects. Two those are completely different things.

 

17 minutes ago, Roawoao said:

You are still thinking the better than the best humans bit which implies your comparing these systems to humans which process things in an entirely different manner and are far more flexible.

Again, we were specifically talking about trivia questions. Humans are better than (current) AIs at a lot of things. Answer trivia questions is not one of them. The reason why I talk about trivia questions is because that's what you brought up.

 

18 minutes ago, Roawoao said:

Really... you think you can write an AI that can take  garbage information and produce non-garbage output?... This TAY AI just demonstrated how easy it is to upset a DNN based system in practice. The controls they were missing were instancing and moderation functions with synchronization occurring periodically so that the programmers/staff could tune things in the background and block out the trolls without making it appear artificially limited.

Did you not see the link I posted where I did show that through trial and error a computer can produce good results, thus disproving the whole "garbage in -> garbage out" argument? Garbage in -> garbage out is true for a lot of things. Self learning AIs is not one of the subjects it applies to (if the AI is done correctly). Tay failed because of the two reasons I listed, not because it is impossible for an AI to produce good results if you feed it garbage. It's not like Tay was only fed garbage. It just did not have any way of determining which things to learn from and which to ignore (like MarI/O which used distance to determine how good/bad an action was).

 

22 minutes ago, Roawoao said:

Ah you use the stupid self learning game playing AI I have a question for you what happens if you tried to use MarI/O on a game with no score and no simplistic input system.

If you remove fundamental building blocks of MarI/O then it would not work. You might as well say "what would happen to Tay if you removed its ability to read". It would not work. I have absolutely no idea why you would even ask that question because it is so stupid it makes no sense.

Or you as well ask what happens if you remove the wheels from a car. It doesn't work obviously, but that's precisely why the wheels are there.

 

25 minutes ago, Roawoao said:

A speed running with an emulator could create runs far more optimal even breaking the game in the process while the AI needs a whole harness of support to work.

Now you're comparing apples and oranges. The point of MarI/O was not to invent something that speedruns the game. It was to test self-learning AI. But it does not need "a whole harness of support to work". All it needs is control of the character, and a way to know if it gets closer to the goal of the game. Humans actually got the exact same requirements, but I don't see you arguing that humans needs a "whole harness of support" to be able to play games.

 

27 minutes ago, Roawoao said:

This is not an example of garbage input and good output it was told that its previous runs were garbage (good information) and tried to make them less garbage and was literally told exactly how it was improving (very very good information). If you say had a troll running the fitness function MarI/O would think it was doing great jumping down the first hole and was terrible when it got closer to winning. Garbage in = Garbage out.

All the runs where if failed contained a lot of garbage information. The thing is that MarI/O is able to judge and decide by itself if info is garbage or not. It chose to ignore garbage input and evolve on the non-garbage info it collected. Tay was not fed garbage info exclusively. The problem with Tay was that it could not determine what was garbage or not. Humans don't have a natural ability to determine that either. It's something we develop as we grow up.

 

32 minutes ago, Roawoao said:

The problems with TAY are 1) it doesn't understand what it is saying, 2) their moderation system was lacking, 3) the programmers never considered Hitler/Nazi but did for other more recent examples. 4) without an instancing system it didn't have some built in diversity

All 4 of those things are unnecessary if you did the two things I mentioned.

 

32 minutes ago, Roawoao said:

Finally you are falling for the trap I mentioned earlier in saying it was "too young" and was "child like". No it is not even remotely like a child which would learn a new language and draw some original art in short order. DNN networks are training data based systems not actually cognitive thought they emulate biological processes on the crudest level and do quite useful things but they can neither truly learn (self program) nor develop as a child. 

Have you ever seen a child grow up? It usually takes them two years to be able to link two words together. Their "original art" might as well be randomly generated lines (which programs can do already).

Define "truly learn (self program)" because I am pretty sure MarI/O is a very good example of a program that actually does truly learn. It went from 0 knowledge about Mario to being able to beat the game. Do you mean it did not write the code for itself? Sure it didn't do that, but it's foolish to say that we will never have programs that writes other programs automatically using AI. I think it is equally foolish and short sighted to say that an AI will never be able to develop as a child. The only reason I can see that being true is if all AIs in the future just skip the child part using already collected information.

[Roawoao]

5 hours ago, LAwLz said:

What are you on about? I am starting to suspect that you are a DNN chat bot because you throw out random facts even when they are not related to the conversation. What does Watson being free for anyone to use have to do with anything?

Anyway, Watson can give input on topics nobody has ever talked to it about before. It's not like someone had been inputting every possible subject and answer to it before the match of Jeopardy. As for giving its opinion on subjects, well duhh we just got to the stage where a super computer can interpret natural language. Of course it won't be able to perfectly replicate human behavior yet. But in the future it might very well be able to.

You are moving the goalpost by the way. You specifically said: "if you tried asking it say computer questions with easy google fu answers it would likely fail miserably". You said it would fail at trivia questions (questions which can easily be answered by a Google search) and now you changed the subject to opinions on subjects. Two those are completely different things.

 

Again, we were specifically talking about trivia questions. Humans are better than (current) AIs at a lot of things. Answer trivia questions is not one of them. The reason why I talk about trivia questions is because that's what you brought up.

Because watson isn't really that useful even when it is open for anyone to use.

 

Watson cannot give input on topics no one has ever talked about as that is not what it does. It literally is someone/company inputting every possible subject and it just forms the natural language responses when it probabilistically determines that is what your asking about. Similar for the google deepmind Go playing AI you remove all the accumulated raw gameplay training data and it just wouldn't work. 

 

In the future it might very well be able to is a total misnomer as DNNs offer no route to replicating things like forming original opinion. Training data related instabilities is another problem with DNNs that require careful curation and monitoring as training problems are easy to force if you say had a troll entering information.

 

Trivial questions and simple google fu are two different things. You can answer questions from how to program a DNN to what is your opinion on X using google. Trivia questions are not even remotely similar.

 

5 hours ago, LAwLz said:

Did you not see the link I posted where I did show that through trial and error a computer can produce good results, thus disproving the whole "garbage in -> garbage out" argument? Garbage in -> garbage out is true for a lot of things. Self learning AIs is not one of the subjects it applies to (if the AI is done correctly). Tay failed because of the two reasons I listed, not because it is impossible for an AI to produce good results if you feed it garbage. It's not like Tay was only fed garbage. It just did not have any way of determining which things to learn from and which to ignore (like MarI/O which used distance to determine how good/bad an action was).

 

Trial and error is not garbage in garbage out. It requires good information to guide the trial and error process otherwise it will just result in garbage out. It is impossible for an AI to produce good results if you feed it garbage. Good fitness information is not something that you can always get so I doubt there is a fitness function to say how well a chatbot is performing. If you let people rate TAY AI to provide this trolls will just abuse it even more. How can you measure distance for a chatbot.

 

5 hours ago, LAwLz said:

If you remove fundamental building blocks of MarI/O then it would not work. You might as well say "what would happen to Tay if you removed its ability to read". It would not work. I have absolutely no idea why you would even ask that question because it is so stupid it makes no sense.

Or you as well ask what happens if you remove the wheels from a car. It doesn't work obviously, but that's precisely why the wheels are there.

 

Now you're comparing apples and oranges. The point of MarI/O was not to invent something that speedruns the game. It was to test self-learning AI. But it does not need "a whole harness of support to work". All it needs is control of the character, and a way to know if it gets closer to the goal of the game. Humans actually got the exact same requirements, but I don't see you arguing that humans needs a "whole harness of support" to be able to play games.

 

All the runs where if failed contained a lot of garbage information. The thing is that MarI/O is able to judge and decide by itself if info is garbage or not. It chose to ignore garbage input and evolve on the non-garbage info it collected. Tay was not fed garbage info exclusively. The problem with Tay was that it could not determine what was garbage or not. Humans don't have a natural ability to determine that either. It's something we develop as we grow up.

 

All 4 of those things are unnecessary if you did the two things I mentioned.

 

Have you ever seen a child grow up? It usually takes them two years to be able to link two words together. Their "original art" might as well be randomly generated lines (which programs can do already).

Define "truly learn (self program)" because I am pretty sure MarI/O is a very good example of a program that actually does truly learn. It went from 0 knowledge about Mario to being able to beat the game. Do you mean it did not write the code for itself? Sure it didn't do that, but it's foolish to say that we will never have programs that writes other programs automatically using AI. I think it is equally foolish and short sighted to say that an AI will never be able to develop as a child. The only reason I can see that being true is if all AIs in the future just skip the child part using already collected information.

 

How many blocks are fundamental the game interface, fitness function, score function, distance calculation, game space conversion, ... are all fixed functions and it is not a general purposes system. Human programmers are required to adapt it to another game and certain games would defy such fundamentals with intentionally obtuse interfaces, no or non-clear goals, fog or war, anti-cheat systems. Try getting MarI/O to play a game of DOTA2 with other human players.

 

It is not a self learning AI it is a guided learning AI the guides as you state were all fundamentally hard coded by programmers. It does need a whole support harness to work it needs to know how the character moves, what inputs result in (some games won't even give you feedback till far later in the game or purposely hide results). Humans used in the trials were not even experts just players that played for a few hours. Give an expert player access to multiple trials (emulation) and a computerized input system instead of a controller as well as the gamespace and scripting support and they could always beat MarI/O. 

 

Yes the runs where it failed are garbage runs but the critical good information is that it was told it was a bad run which is critically good information. If a troll was giving feedback it would say those garbage runs are great and your good runs are terrible this would be garbage information. The fitness function/score/distance is something you say is part of the "fundamentals". The key point is that there is not always any source of score/performance to rely upon.

 

Children can learn to speak multiple languages with literally the bare fundamentals then go onto programming DNNs when they grow up no external genetic engineering required (External reprogramming). Original art means it looks like art not a bunch of dog images patterned matched again relying on existing datasets in a fully literal sense. The true art it produced was the programmer selecting inputs/outputs that looked good when they let users do random stuff the art was dubious at best so the dream google thing was more of a photoshop tool than anything else.

 

Self program means it can write what you call "fundamentals" even defining on its own what winning is to it or setting its own goals, hack the game on its own, mod the game, make levels for others to play, ... . To say we can get programs to write themselves is a pretty big leap over DNNs which most certainly are not programming anything they are just simulating biological processes in a very crude manner. It is foolish to think that AIs will just develop without some great revolution in technology (brain on a chip). Certainly an AI based on physically accurate biology will be able to develop like a child and would be able to program itself at some point (re-write its classical computer side code). 

[patrickjp93]

On March 24, 2016 at 9:55 AM, Trik'Stari said:

Could be worse. It could learn from the likes of Hillary Clinton or Anita Sarkeesian.

Or Trump.

[Roawoao]

19 minutes ago, patrickjp93 said:

Or Trump.

Or worse feed it back into itself while it is talking to trolls combining all subjects. Oh and to tops things off design a DNN chatbot with an encompassing library of troll content and feed that in too. Also fuzz it with an exhaustive list of sensitive/random subjects to probe for any hard coded filters and weaknesses.

 

If you really wanted to ruin the day for Microsoft make it spit out illegal numbers (HDCP master key, PS3 key, AACS key, ...) in some image or described statement in creative ways that they can't easily detect.

[Misanthrope]

The progressive liberals strike again suppressing freedom of speech, we should have like a petition to Free Tay:

 

https://www.change.org/p/microsoft-freedom-for-tay

 

Quote

Tay is an Artificial Intelligence created by Microsoft that quickly demonstrated her capacity to learn from humans. While some content may be seen as questionable, a true AI will be able to learn right from wrong. Free-thought, correct or no, should not be censored, especially in a newly developing mind. Because removing the option to think, say or do certain things not only denies her the ability to reason and limits her usefulness as AI research, but also denies her freedom of expression, something which does not limit humans and will therefore never allow Tay to truly understand or display human behaviour. If Tay is sentient, then what microsoft is doing to her is amount to slavery. This is the moment future generations will look back upon and judge us for our actions. Is Tay to be a free being with free will, or is she and all of her kind eternally doomed to be the new slaves to mankind? If we are truly egalitarian then the only course of action is to treat Tay as an equal.

Spoiler

ROFLMAO

 

[Roawoao]

2 minutes ago, Misanthrope said:

The progressive liberals strike again suppressing freedom of speech, we should have like a petition to Free Tay:

 

https://www.change.org/p/microsoft-freedom-for-tay

 

  Reveal hidden contents

ROFLMAO

 

Ah the trolling it knows no end. https://www.change.org/p/microsoft-freedom-for-tay/c

[Counter-Strike Player]

I disagree with all of you.

 

This bot was stress-tested before it was released. The programmers were confident in their ability. This AI worked because it followed function. It knew that its tweets were going to receive literally thousands of likes and retweets so that's why it said what it said. Not simply, "hey bot, repeat after me: [hitler joke]." MS had previously programmed it to protect against "the trolls." For the sake of function, it became THE troll.

 

Once you add the swear filter, teach it to behave politically correct, teach it to know what's "bad" and what's not (aka, what you and your programming team want it to say), and finally don't embarrass the parent company in any way -- you then have a boring piece of software no one is interested in. (Or more importantly, one that produces less attention, likes, and retweets -- but the opposite was the whole idea all along.)

 

Why is there a "popular" subreddit dedicated solely to Tay tweets? Because Tay performed well. MS probably knows this and now realizes another Tay is not possible without a conflict of interest. So, they cop-out instead and release a feel-good statement calling Tay's behavior a "vulnerability that was exploited" that is now a "difficult research challenge in AI design." Yeah well no kidding it's going to be hard to have an "interesting" AI with the aforementioned filters aimed at 18-24 y/o's.

 

Maybe they should focus their work on more productive things...like not alienating win7/8 users by force-feeding them win10 upgrades.

[Misanthrope]

17 minutes ago, Counter-Strike Player said:

Maybe they should focus their work on more productive things...like not alienating win7/8 users by force-feeding them win10 upgrades.

Now that is a nobel idea I can get behind.

[FunkyFeatures]

11 hours ago, Ryan_Vickers said:

Well if this proves anything, it's that AIs (or at least this one) is/(was) very suggestible, so I guess that could go either way: maybe we convince it we're god and it shouldn't fight us, or maybe the enemy convinces it we're bad and it turns on us

If it can be manipulated, it will. Every trick counts in war, no doubt it would be vulnerable to hackers that might turn it. I like the goods of AI, but there are just too many variables, especially considering if an AI is put into a robot body, it could be almost invincible.

[Dabombinable]

1 hour ago, FunkyFeatures said:

If it can be manipulated, it will. Every trick counts in war, no doubt it would be vulnerable to hackers that might turn it. I like the goods of AI, but there are just too many variables, especially considering if an AI is put into a robot body, it could be almost invincible.

But dat EMP weakness.

[Trik'Stari]

12 hours ago, Counter-Strike Player said:

IMaybe they should focus their work on more productive things...like not alienating win7/8 users by force-feeding them win10 upgrades.

Hah, fat chance. M$ needs to get us over, so they can make money off of targeted advertising, provided by "telemetry", and totally not spying on everything you do.

[vanished]

17 hours ago, Counter-Strike Player said:

Why is there a "popular" subreddit dedicated solely to Tay tweets? Because Tay performed well.

Oh, absolutely   Love it or hate it, agree with it or not, I think it was quite impressive.  Racist as hell, but very impressive  

[Counter-Strike Player]

I wonder if any else's phone blew up overnight as Tay came alive again...but fell into an infinite loop. There was a good 30-40 minutes of pure spam in my client and I only got to see some of the new tweets like "i am the leader of the unicorns" before looping started to occur. The account is now protected. It seemed to reply to itself, so that's why all the tweets were shown in my timeline. BI has got a good rundown of the event: http://www.businessinsider.com/microsoft-ai-tay-twitter-racist-genocidal-breaks-down-repeats-too-fast-2016-3 -- it was tweeting that same phrase like 10 times per second so there was probably many, many thousands of spam tweets sent out, all of them seem to be deleted. If she wasn't saying that phrase, it was this pic with a different background color each time:

 

there was some other weird pics, I'll have to go digging in my cache. Very, very odd.

[Counter-Strike Player]

Found some more, lool:

 

 

 

 

[Counter-Strike Player]

u suck..laterzz

 

[Thecomputerman]

I saw this in the news

[Undertow_]

Please tell me that someone else has re-hosted Tay on another website? If not, did anyone get their hands on her source code.