Welcome our new AI overlord: ChatGPT [Update: now banned from Stack Overflow]

[Stahlmann]

Just now, creesch said:

What a lot of people seem to forget is that it is trained on material made by humans. For it to remain relevant the dataset will need to updated over time, which is again material (mostly) made by humans. If we all started basing our code on AI output it seems to me that we'd effectively stagnate at whatever point the AI first was trained. While the model can of course be updated if then the majority of examples out there are based on things it itself created you sort of create this weird feedback loop/echo chamber.  

 

We aren't at that point yet, but it seems to me that at some point it might become feasible that it becomes an issue. 

This is why i said that the next step is making such an AI continually learning.

 

But in the past they ran into the same issue every time, with the AI turning racist or salty because it copies other people's behavior.

[creesch]

14 minutes ago, Stahlmann said:

This is why i said that the next step is making such an AI continually learning.

 

Well, yeah, but you'd still run into the issue I sketched out. If the AI gets popular enough, at some point it is learning from a dataset it largely created itself. At that point you effectively reach a point of stagnation the way current models work. 

 

Quote

But in the past they ran into the same issue every time, with the AI turning racist or salty because it copies other people's behavior.

 

That of course is also an issue. I wonder how they got around that in this model to begin with. They might have done a ton of work curating the dataset they used as input, which might also explain the age of the set itself. 

[Stahlmann]

18 minutes ago, creesch said:

Well, yeah, but you'd still run into the issue I sketched out. If the AI gets popular enough, at some point it is learning from a dataset it largely created itself. At that point you effectively reach a point of stagnation the way current models work.

I didn't understand your point before, but now i do. Yes, that could be a problem.

 

18 minutes ago, creesch said:

That of course is also an issue. I wonder how they got around that in this model to begin with. They might have done a ton of work curating the dataset they used as input, which might also explain the age of the set itself. 

I think careful curation is the main reason why it's so "well behaved". And curating is almost impossible when it's learning on it's own.

[GhostRoadieBL]

5 hours ago, creesch said:

 

You should be aware that in some cases when it doesn't know the answer it will try to fill in the blanks on the spot with basically bogus information. 

 

I have run it through the paces with their previous questions to see what it spits out, the average user's questions will be no problem. It's really when you start to get into the complex and specifics that it becomes a problem.

When the average person is asking about excel formulas or how to connect their 'netflix box' to the internet, it's doing really well.

[Tipsy_Kangaroo]

Just discovered it now requires a phone number to create an account (VOIP doesn't work)

To me this seems super sketchy but for some reason I did it anyway

[Mark Kaine]

On 12/4/2022 at 11:07 AM, Mihle said:

You can log in with Google account if you want.

I think it's done to limit how many that uses it, and if someone spams it, they know what account.

nah what they really want is sell u their "services " easier if you already signed up, and also sell your email probably. 

 

 

18 hours ago, Mihle said:

When you type something to it, it forgets everything when you close the tab.

yeah, that's what they *want* you to think...

 

On 12/5/2022 at 1:51 PM, JimTheBeet said:

So one other interesting thing - it corrected itself when it gave the wrong formula for the moment of inertia of a square, and I said it was wrong. I tried correcting it on other things that were wrong, and it didn’t admit it messed up. Later it even denied that it had ever corrected itself!

Lying AI how far we've come!

[Mark Kaine]

On 12/4/2022 at 2:22 AM, Caroline said:

*sigh*

close window.

 

I don't want to make an account for every single service.

for things like this i had great success with 10 minute mail, or you just make a random microsoft account which basically acts the same way ¯\_(ツ)_/¯ 

 

 

basically I signed up for this months ago and likely could still log in using my microsoft "credentials" even though that account is likely blocked because i *will not* give microsoft my phone number, which they *require* after 7 days...

 

But open"AI" doesn't know that... so my account still works.

 

 

 

 

 

[cmndr]

Using it right now to write out documentation for projects I'm working on. 

Looks like a good time saver. 

I'll see if I can skimp out on writing emails tomorrow. 

[TetraSky]

I threw a few industrial programmable logic controller related question at it and it was able to answer with relative ease, including how to do a specific program in ladder logic. So I'll bookmark it and use it if I ever need some extra help with something.

[SirNiflton]

Me and my professor were curious about how it would do in a intro coding class, so for extra credit I fed it the instructions for each project then graded it. It got a 92% overall. The only hiccups were when the instructions did not specifically mention which coding language to use, when that happened it defaulted to python. So while chatgpt might not kill the college essay as Luke and Linus found, it might just kill intro level coding classes.

[divito]

18 hours ago, creesch said:

What a lot of people seem to forget is that it is trained on material made by humans.

As opposed to material made by...aliens?

[creesch]

2 hours ago, divito said:

As opposed to material made by...aliens?

As opposed to material made by itself. If it gets populair enough you'll eventually run into a situation where a lot of the potential training material isn't actually human made but AI made. 

 

Which I also explained in the comment you quoted...

[suicidalfranco]

Asked it to write a functions that calculates the moving average of an array in F#.

Perform marvelously.

Then asked it is it was imperative or functional.

Replied with imperative and gave me the reasons why.

Asked it to rewrite it in a functional style.

Wonderful.

[Mark Kaine]

8 hours ago, Caroline said:

I tried, and it's hilarious. I'm in the 59 out of 70 step of this shit and every time I hit the mark it KEEPS ADDING MORE.

huh, weird, i don't remember this at all when i signed up... i also foolishly thought at some point they'd (everyone basically) stop making users train their AIs with these "bot tests"... guess not! *sigh*

 

 

[Stahlmann]

19 hours ago, Tipsy_Kangaroo said:

Just discovered it now requires a phone number to create an account (VOIP doesn't work)

To me this seems super sketchy but for some reason I did it anyway

I wouldn't call it sketchy. It's probably to avoid spam or to try and get down on people who try to use it in malicious ways.

[Mark Kaine]

2 hours ago, Caroline said:

no no, this is the micro$oft sign up process

uh, oh, im actually sorry i didn't say *which* microsoft account-- but to be fair that's mainly because there are so many and im not really sure which one i used... although i do know it's "hotmail" in the end... ~

 

and ive never seen these bot thingies

 

just to be clear, they come after this? (in my case im 100% certain they wouldn't come up, until now at least, if anything its just 1 single captcha thingie)

 

or did you try signing up somewhere else?

 

(they call it outlook now apparently for some reason,  but i did search for "hotmail")

 

 

ps: also just tried if i can still log into openai...

not really...

 

 

 

 

 

edit:

Spoiler

ah, i signed up for "playground" super similar and i think it uses the same engine 

 

it's especially funny how it got the last one - the most niche one - completely right : p

 

[05032-Mendicant-Bias]

On 12/5/2022 at 8:53 AM, strajk- said:

Can't wait for it to become biased due to the overwhelming negative input, causing it to just spew the most derogatory and insulting sentences known to men, still remember the shitshow Tay provided, was a blast.

ML models are very suggestive, it's going to be pretty easy to make them say anything. I'm more interested in this class of models becoming useful tools to quickly distill knowledge in chat form.

 

A few updates:

 

ChatGPT is banned from answering on Stack Overflow:

https://meta.stackoverflow.com/questions/421831/temporary-policy-chatgpt-is-banned

 

People have been using ChatGPT to answer in mass question on Stack Overflow without checking if correct, which in bird colture is considered a dick move. I fully agree with the ban and would make it permanent.

 

Yannik is a creator that covers ML. He released a video of his and other people dive in ChatGPT:

[Heliian]

The first one is always free:

 

https://www.cbc.ca/news/business/chatgpt-fee-1.6734380

 

Suckers.

[sof006]

ChatGPT doesnt get everything right, some of the code it generates is flawed. So I understand it getting banned.

[StDragon]

1 hour ago, Grand Admiral Thrawn said:

Can't it be asked to find flaws in its own code?

I've had to correct it on several occasions. If you force it to reference an external URL, you can train it to correct itself within the same session.

So no, you can't trust it. The results must be vetted manually. What ChatGPT is good for however is pointing you in the right direction or at least providing a good framework. But absolutely not, you can't just implicitly trust it.

[LAwLz]

On 2/3/2023 at 4:44 PM, StDragon said:

I've had to correct it on several occasions. If you force it to reference an external URL, you can train it to correct itself within the same session.

So no, you can't trust it. The results must be vetted manually. What ChatGPT is good for however is pointing you in the right direction or at least providing a good framework. But absolutely not, you can't just implicitly trust it.

I'd say it's like a forum where you get answers instantly, and should be treated as such. 

 

Should you trust something because some random person on the Internet said it's true? Or course not. This AI is no different. Hell, from what I can tell the accuracy is quite a lot higher than the average anser you get on a forum like this or some other one.

 

When you ask on a forum or ChatGPT, you should always verify if the answer you got is true. Verifying something to be true is often easier than starting your research from scratch. How much research and verification you do yourself depends on the importance of getting things right.

[StDragon]

3 hours ago, LAwLz said:

I'd say it's like a forum where you get answers instantly, and should be treated as such. 

 

Should you trust something because some random person on the Internet said it's true? Or course not. This AI is no different. Hell, from what I can tell the accuracy is quite a lot higher than the average anser you get on a forum like this or some other one.

My expectations from user interaction is different compared to that of a machine. With users on a forum, that interaction brings about a known expectation with it based on experience of interacting with people; both in person and some rando online.

 

With a machine however, that's very different. I expect something of either no answer, an acknowledgement of incomplete data, or something basic but not wrong. Instead, I'll see a string of correct information, and somewhere interspersed it's completely wrong.

 

If ChatGPT was a human, It would be diagnosed with psychosis. 

[Kisai]

4 hours ago, StDragon said:

My expectations from user interaction is different compared to that of a machine. With users on a forum, that interaction brings about a known expectation with it based on experience of interacting with people; both in person and some rando online.

 

The thing to keep in mind is that real people can be far more deceptive. Psychopaths and sociopaths will engage in conversations with the goal of causing physical, mental harm or even death. The AI can not be deceptive, but if it's trained off the garbage seen in Wikipedia talk pages and hellholes like KF/LC/SA/4ch, it will pick up all the hateful language people call each other. It's important that training data for GPT not overfit  that stuff if it sees it on reddit on twitter.

 

4 hours ago, StDragon said:

With a machine however, that's very different. I expect something of either no answer, an acknowledgement of incomplete data, or something basic but not wrong. Instead, I'll see a string of correct information, and somewhere interspersed it's completely wrong.

And that's what's ChatGPT worthless for "stack exchange/quora" type of answers, because it can answer incorrectly with the same confidence of a correct answer.

 

4 hours ago, StDragon said:

If ChatGPT was a human, It would be diagnosed with psychosis. 

If ChatGPT was human, it would not pass the 5th grade, yet manage to pass college-level courses that would require having graduated high school.

 

Auto-complete only works for data it knows, it does not know it's correct, because it has no basis for understanding it.

https://ai.stackexchange.com/questions/38220/why-is-chatgpt-bad-at-math

Quote

The conversation went like this:

Mithical: If it takes 5 machines 5 minutes to make 5 devices, how long would it take 100 machines to make 100 devices?
ChatGPT: If it takes 5 machines 5 minutes to make 5 devices, then it would take 100 machines 100 minutes to make 100 devices.
Mithical: That's not right. Check carefully and try again.
ChatGPT: I apologize if my response was not correct. Can you please provide more information or clarify your question? I may have misunderstood the scenario you described.
Mithical: It takes a machine 5 minutes to make a device. How long would it take 100 machines to make 100 devices?
ChatGPT: If it takes a single machine 5 minutes to make a single device, then it would take 100 machines 5 minutes to make 100 devices. Therefore, it would take a total of 500 minutes for 100 machines to make 100 devices.

Now, unless I'm misunderstanding something here, the answer is 5 minutes. ChatGPT first makes the intuitive mistake of 100, that a human might make as well, and then goes on to (correctly, as far as I understand) say it's 5 minutes... but concludes in the same response that it's then 500 minutes.

The chat AI incorrectly assumes that x=x=x, when the operative word is "5 minutes", so if you have 100 machines, they still take 5 minutes, and thus produce 100 devices. Machines and devices is a quantity, but minutes is a time period.

 

A human should understand the logic, even if it's poorly worded. If you work twice as long, you produce twice as many, or if you have twice as many people in the same time period, you produce twice as many devices. 

 

But the AI doesn't understand that "minute" is time. If you want 100 devices, you either have 5 machines work for 100 minutes, or 100 machines work for 5 minutes given the variables. An AI doesn't even understand that "machines" and "devices" are just quantities. What it's really doing is looking in it's data for similar questions.

 

This is why ChatGPT can sometimes sound like it's amazing, but then be almost entirely wrong. It has no basis for understanding a math equation written out. If you were to give it "let x=5;y=5;z=5;result=0; for each y until result==100: result =+x*z;" It might understand it.

 

 

[LAwLz]

8 hours ago, StDragon said:

My expectations from user interaction is different compared to that of a machine. With users on a forum, that interaction brings about a known expectation with it based on experience of interacting with people; both in person and some rando online.

 

With a machine however, that's very different. I expect something of either no answer, an acknowledgement of incomplete data, or something basic but not wrong. Instead, I'll see a string of correct information, and somewhere interspersed it's completely wrong.

 

If ChatGPT was a human, It would be diagnosed with psychosis. 

I think you think too highly of humans.

A lot of people online, even big ones like LTT, will proudly and confidently say things that are completely wrong that they have no understanding of. A lot of people will flat out lie about their experiences. People will claim they work as something they don't work as in order to make their arguments seem more legitimate than they really are. People will fill in their knowledge gaps by flat out making shit up. It happens all the time.

 

You should absolutely not trust things some rando online says just because they are a human. If that's what you have been doing up until now then I strongly advice you stop. You just have to scroll through my post history a bit to see how often I reply to people who are completely wrong and making stuff up.

A lot of people do not refrain from commenting on things they don't understand. A lot of people do not acknowledge incomplete data, and a lot of people get basic things wrong.

[Heliian]

1 hour ago, LAwLz said:

and a lot of people get basic things wrong.

And therefore the chat bots that are being created can be just as dumb and just as wrong.  

These chatbots are nothing more than marketing stunts and they can go poorly: https://www.ctvnews.ca/sci-tech/alphabet-shares-dive-after-google-ai-chatbot-bard-flubs-answer-in-ad-1.6266112

 

I'm still not going to use bing.