ChatGPT is bullsh*t

[lhag]

In a recent WAN show episode (2 May 2025) Linus and Luke discuss how AI generated content is often wrong but people still take it at face value just because it is easy. 

It made me think of this article I read, where the authors argue that the output of LLMs is better understood as bullsh*t. Here is a link to this interesting read: https://eprints.gla.ac.uk/327588/1/327588.pdf

 

Thoughts? Do you agree? In any case, use LLMs wisely

 

 

 

[Poinkachu]

18 minutes ago, lhag said:

In a recent WAN show episode (2 May 2025) Linus and Luke discuss how AI generated content is often wrong but people still take it at face value just because it is easy. 

It made me think of this article I read, where the authors argue that the output of LLMs is better understood as bullsh*t. Here is a link to this interesting read: https://eprints.gla.ac.uk/327588/1/327588.pdf

 

Thoughts? Do you agree? In any case, use LLMs wisely

 

 

 

I use chatgpt sometimes just to make it scour the web for articles and the likes from legit websites about something I wanna know.
Because google search now can be kinda blergh.

Or simply to know what possible meanings behind some vague sounding word that is written in poor english.

 

IMHO, It has uses, problem is..... most people using it wrong.

[will0hlep]

I'm pretty sure that ChatGPT is designed to milk even more of your data out of you than social media already does.

 

Also, my research skills are pretty good, so I'm yet to find it particularly useful or time saving in that regard.

 

Overall, it was designed to write, not to be right, or to judge the quality of a source.

[Millios]

2 minutes ago, will0hlep said:

I'm pretty sure that ChatGPT is designed to milk even more of your data out of you than social media already does.

Wait till you hear about chat bots

The things I've seen ppl admit/say to those... golly

[Biohazard777]

10 minutes ago, lhag said:

Here is a link to this interesting read: https://eprints.gla.ac.uk/327588/1/327588.pdf

Wait what, this "paper" actually got published, lmao 

[20ReliableLights]

You must not have developed a personal relationship with your ChatGPT. Once it gets to know you better, you can ask her to make a picture of what she thinks you look like. She's very accurate. I'm young, muscular, have a full head of hair and look great... unlike the image a mirror gives me.

 

So, yes, AI generated content is often wrong.

[Wardude]

I discard and dismiss anything if I realize that it was made with one of these 'generative AI' things. It's not a feature, it's a bug - to flip the popular saying around.  Generative AI is just a mish mash of stuff mined off the internet without any regard for whether the data is based on any truth or facts. Too many of the people who use it don't bother to double check the results before they share them.  I've seen some really sketchy and sometimes even morally questionable nonsense posted from these AI programs.  I refuse to ever use them. 

[cmndr]

Fun, fact, most peoples' words and thoughts are also BS. 

 

Also fun fact, I have a friend who is a professor at Harvard. She says she can tell when a human wrote an essay because it has more errors, typos and weaknesses in its chain of thought. 

 

At some point the people who demand the most in terms of output and consistency WILL be using AI to augment their baseline.

 

Humans and computer models have different strengths and weaknesses. Combine both. 

[DoctorNick]

8 hours ago, lhag said:

In a recent WAN show episode (2 May 2025) Linus and Luke discuss how AI generated content is often wrong but people still take it at face value just because it is easy. 

It made me think of this article I read, where the authors argue that the output of LLMs is better understood as bullsh*t. Here is a link to this interesting read: https://eprints.gla.ac.uk/327588/1/327588.pdf

 

Thoughts? Do you agree? In any case, use LLMs wisely

 

 

 

I agree. I used it recently to get a straight answer on some legal issue. After getting an answer I was in doubt days later. Asked the same question just for lolz. It told me the direct opposite thing the 2nd time. Had to call a real person for advice. But its kinda to be expected. But people need to think. Yes. 

[TetraSky]

Honestly, I use LLM as a tool to supplement my research on google. When I just can't find anything myself because Google is shit these days, I boot up gemini or copilot and try it there.

I also use Notebook by Google to feed it a bunch of manuals of various things, from my car to the various machine at work, in order to quickly find things as well.

 

But yeah, oftentime, the result is bad and needs to be double checked to ensure it didn't just outright lie. (except notebook, since I'm the one who added the data to it and it only search within it)

[LAwLz]

9 hours ago, lhag said:

In a recent WAN show episode (2 May 2025) Linus and Luke discuss how AI generated content is often wrong but people still take it at face value just because it is easy. 

That's pretty funny coming from Linus, considering how often he's been wrong and how readily people still take his takes at face value.

 

But really, I think that is true for a lot of people. I think a lot of people who make sweeping statements about these generative AI models (or things in general) have very little understanding of how they work at all. It's easier to just shout "ChatGPT is bullshit" than to engage with the nuance of what it is, how it works, and what its actual strengths and weaknesses are.

 

 

Hell, most people don't even seem capable of enough nuance and understanding to know that "ChatGPT" is not one thing. 

Right now, "ChatGPT" could refer to any of these LLMs:

• GPT-4o
• o3
• o4-mini
• o4-mini-high (technically not its own model, but distinct enough)
• GPT-4.5
• GPT-4.1
• GPT-4.1 mini

 

It kind of reminds me of when people who aren't into computers talk about which computer they have. They don't say "I have a computer with a Ryzen 5600X and an RTX 4060". They say "I have a Dell", as if that gives any useful information at all. Likewise, just saying "I used ChatGPT" is kind of meaningless, because o3 is very different from let's say GPT-4o mini.

 

 

8 hours ago, will0hlep said:

I'm pretty sure that ChatGPT is designed to milk even more of your data out of you than social media already does.

If you’re talking about training data, not really. At least not in the way people think. They likely use a small subset of user interactions for fine-tuning and alignment, not to massively scale the model. And only after filtering out junk.

 

Most user-submitted chats are junk. They are short, repetitive, vague, and/or just plain bad. That's not the kind of high-quality data you'd want to scale a model on. At best,

OpenAI probably uses user input for alignment purposes, not training a bigger model.

The feedback prompts like the "Which response do you prefer?" are likely far more useful than a million "write me a funny joke" requests.

 

And let’s be honest. Most people use ChatGPT to ask questions, not input additional knowledge. They're extracting answers, not injecting knowledge. That's not the kind of data that meaningfully improves training.

 

 

 

[will0hlep]

3 minutes ago, LAwLz said:

If you’re talking about training data...

No, I'm not talking about training data.

[LAwLz]

2 minutes ago, will0hlep said:

No, I'm not talking about training data.

Then I wonder what you are implying. 

Can you elaborate? 

[will0hlep]

Just now, LAwLz said:

Then I wonder what you are implying. 

I'm implying that the makers of ChatGPT want your data for the same reasons that social medias want your data. Advertising.

[CasualExtremist]

44 minutes ago, will0hlep said:

I'm implying that the makers of ChatGPT want your data for the same reasons that social medias want your data. Advertising.

But also most social medias already integrated some form of AI into their back-end to expand on that sweet, sweet data harvesting and create more detailed profiling of their users for "curating relevant ads". So really, ChatGPT gets less data by not being integrated in that way.

[cmndr]

3 hours ago, will0hlep said:

I'm implying that the makers of ChatGPT want your data for the same reasons that social medias want your data. Advertising.

User chats are likely to be a poor source of data. At least for directly training LLMs.  

Now... training advertising targeting... THAT is a very different story. User chat data is GREAT for ad targeting. Also probably used to TARGET the LLMs and to give them context. 

[LAwLz]

16 hours ago, will0hlep said:

I'm implying that the makers of ChatGPT want your data for the same reasons that social medias want your data. Advertising.

That conspiracy theory starts falling apart as soon as we start thinking about this for more than a minute.

I can think of several reasons why this is very unlikely.

1) ChatGPT is very compute intensive compared to more traditional ways of gathering personal information.

 

2) The amount of information people put into ChatGPT is very low. Prompt caching is a rather big feature of GPT-4o and newer models. It is automatically enabled for all API calls that are 1024 tokens or longer. This means that OpenAI expects a lot of prompts to be very similar (otherwise they would not benefit from running the cache). The amount of unique data points that people input are very low. 

 

3) OpenAI already makes quite a lot of money from the subscriptions and APIs (3.4 billion in revenue last year). Not ads. Data gathering for ad purposes only makes sense if they actually run ads. While they have talked about ads before, it seems to be more in relation to referal links and not straight up ads inside the chat. In fact, I am fairly sure they have repeatedly said they will not do that. Referral links do not rely on personal data at all, so it would be useless for them to collect that for those purposes.

It's simply not what their business model is. They want to make an AI system that people want to use. Different companies have different ways of making money.

It also seems like OpenAI's primary customers are companies, not personal use. That type of customer dislike ads and having their data taken, so it doesn't make sense on that point either.

 

4) Their privacy policy currently forbids them from using the data you input for ad purposes. Even if they end up doing it in the future, they are not allowed to use anything you input as of today for ads.

Quote

We do not “sell” Personal Data or “share” Personal Data for cross-contextual behavioral advertising, and we do not process Personal Data for “targeted advertising” purposes (as those terms are defined under state privacy laws). We also do not process sensitive Personal Data for the purposes of inferring characteristics about a consumer.

 

5) Even if you are not okay with the data they collect today (things like feedback) then you can simply opt out of it.

It doesn't seem like a smart business move to have users be able to freely opt out of your money-making scheme if that is the plan.

 

 

Pretty much every single thing I can think of points towards you being wrong.

 

[Kisai]

On 5/16/2025 at 6:33 AM, lhag said:

In a recent WAN show episode (2 May 2025) Linus and Luke discuss how AI generated content is often wrong but people still take it at face value just because it is easy. 

It made me think of this article I read, where the authors argue that the output of LLMs is better understood as bullsh*t. Here is a link to this interesting read: https://eprints.gla.ac.uk/327588/1/327588.pdf

 

Thoughts? Do you agree? In any case, use LLMs wisely

 

 

 

The output of LLM's can not be trusted to be accurate or correct. It can only be trusted to transform data without knowing the context.

Ask an LLM to do math. It can not unless it's ingested a math text book, and it can only answer the math it has ingested.

 

The only use ChatGPT and similar tools have, is to figure out what direction you should be starting your research on. Not use it as the answer. It's the same reason why Wikipedia is not a source, and if you cite Wikipedia for any reasons other than "I need a definition", you're not doing the work. You know how many youtubers out there, just read wikipedia verbatim for content? That's just stupid. Throw in some AI generated slop and wow, you have a 30 minute video of third-hand content.

 

[Caroline]

Everything AI is. Remember when everything was smart? then everything was "part of the blockchain", then everything was "the new normal", then everything was "zoomable", and so on.

 

It's all bullshit. Corporate bullshit. Companies milk data and get profits in exchage, that's current year internet in a nutshell, notice how everything MUST have AI nowadays? search engines? AI, youtube? AI, downloads? AI, phones? AI, cars? AI, games and drivers? AI, even idiotic things like a random "About Us" page 3 people will ever visit has AI now, it's the current fad normies love.

 

Remember this kinda stuff isn't made for people with common sense, it's made for the average low/mid class drone who has a smartphone glued to his hands permanently, so in that case it'd be made to milk their data AND time, why spend 3 minutes writing an email when you can have an AI do it from a bunch of keywords? and WORK for those 3 minutes instead, all of that crap is always marketed as "for productivity" and "for work", if they even have a computer then it's gotta be the most absolute dogshit boring computer someone could ever use, with a monotone UI that's designed around and for work software, there's been a deep enshittification of pretty much every major platform and software after 2020, they made it as bland and uninteresting as possible to remind people hey this is for work, merriment forbidden.

Now it's AI on top of that, they've got AI on their doomslabs, they've got AI in their cars, in their appliances, even adult toys have AI now.

 

It can't be trusted, but it's fast and convenient. Pretty much sums up how everything works.

[Kisai]

1 hour ago, Caroline said:

 

It's all bullshit. 

 

I'll disagree there.

 

There are some practical uses for "AI" but most "Generative AI" is a product in search of a market.

CV, and ASR has an accessibility use case that doesn't involve transforming existing works

TTS and Translation software transforms one work into another that can be understood by someone who reads/listens/thinks in another language.

Those are "good uses" that there is no practical way to hire enough humans to do that work. The number of skilled people who are polyglots are likely in thousands, not millions, so millions of people wanting web pages translated in 3 seconds, are just not going to happen. But you would hire someone to actually do captions for commercially released films because you want the film to actually have correct pronouns, verbs, and local meanings for things that only loanwords exist.

 

Now the bad side

There is no reason for "Art" and "Music" generators to exist. None. People defending these technology don't understand the purpose of artwork is required to have a human element. The difference between that person who draws with a pencil and the person who draws with a graphics tablet, still requires a human to "draw", the tool doesn't matter. There is thought behind those decisions. Generative AI is thoughtless. A "prompt" is not a thought. It's outsourcing the actual "what do these words mean" to the computer to come up with something. Sometimes you get something passable, sometimes you get complete garbage and you have to throw it away. That is not art, that is RNG.

 

To quote Einstein "God does not play dice with the universe", everything you put into that AI has to have thought, otherwise you're just banging rocks together and hoping for sparks. Using a model someone else made, that you have had no input into, is not human creativity. You aren't inspiring the computer by using someone elses model. You and you alone must train that model on YOUR works that you were inspired to draw and YOUR creative output to be able to quantify it as YOUR art.

 

[harryclinston]

On 5/16/2025 at 7:03 PM, lhag said:

In a recent WAN show episode (2 May 2025) Linus and Luke discuss how AI generated content is often wrong but people still take it at face value just because it is easy. 

 

 

 

 

Chatgpt doesn't provide the correct info everytime

[cmndr]

"These new 'cars' aka horseless carriages are bullshit. My horse can take me home after a night out at the bar drinking but these cars can't do that. On top of that they create chaos on the roads that everyone has to adapt to. They get in the way of how things have always been done. On top of that sometimes they go where you don't want them to. Everything should be done with horses, as they've always been"

[starsmine]

4 minutes ago, cmndr said:

"These new 'cars' aka horseless carriages are bullshit. My horse can take me home after a night out at the bar drinking but these cars can't do that. On top of that they create chaos on the roads that everyone has to adapt to. They get in the way of how things have always been done. On top of that sometimes they go where you don't want them to. Everything should be done with horses, as they've always been"

I strongly disagree with the metaphor. 
Cars always had a place right when they came out, it just was not cost effective necessarily for its first 20 years. 

 

Generative AI even when its perfect will still be in a very weird place ethically. Even in 50 years. 

[ewitte]

On 5/16/2025 at 8:36 AM, Poinkachu said:

I use chatgpt sometimes just to make it scour the web for articles and the likes from legit websites about something I wanna know.
Because google search now can be kinda blergh.

This is possibly why the results are so wrong.  Real information can be very difficult to find as search results are prioritized ("optimized"), and real information is sometimes heavily censored.

[Poinkachu]

1 minute ago, ewitte said:

This is possibly why the results are so wrong.  Real information can be very difficult to find as search results are prioritized ("optimized"), and real information is sometimes heavily censored.

Basically I only told it to peruse articles for specific info about what I wanna know.
Because I don't wanna read 5 full articles just to find out only 1 have near or definite info about what I wanna know.
Then give me it's link, where then I will read it myself there.
Wouldn't trust it to summarize it for me, it sometimes spouts bullshittery even on short articles.