AI gets US Election information wrong.

[Eric the Grey]

Summary

Study shows that more than a quarter of election information shared by AI factually incorrect

 

Quotes

Quote

More than a quarter of the election information shared by popular AI large language models is factually incorrect

My thoughts

If AI models cannot even correctly tell us how long until the next US election day, or even report the same number of days when asked multiple times, they have a long way to go before they become trustworthy.  

 

According to this, Google does state that Gemini is restricted from election results, but simple factual information like counting the number of days until the a specific date makes me want to ignore any results it may offer.  

 

Sources

MSNBC

 

[WereCat]

In other news... Random number generator gets equation results wrong most of the time. 

[okkee]

2 minutes ago, Eric the Grey said:

Summary

Study shows that more than a quarter of election information shared by AI factually incorrect

 

Quotes

My thoughts

If AI models cannot even correctly tell us how long until the next US election day, or even report the same number of days when asked multiple times, they have a long way to go before they become trustworthy.  

 

According to this, Google does state that Gemini is restricted from election results, but simple factual information like counting the number of days until the a specific date makes me want to ignore any results it may offer.  

 

Sources

MSNBC

 

Is anybody trusting the info they get from AI. Like, it just gets a whole ton of things wrong and it seems like the companies are also trying to avoid privacy suits. in addition to that, it's neutered for some reason and will flatout refuse to output answers for certain topics. like, just dont use it for info.

[Monkey Dust]

AI can't understand its own output. This makes AI validating itself, and therefore producing reliable answers, somewhat challenging.

 

The more of AI I see, the more I think it's 20% useful product, 80% hype. And 20% maybe overly generous.

[WereCat]

Just now, Monkey Dust said:

AI can't understand its own output. This makes AI validating itself, and therefore producing reliable answers, somewhat challenging.

 

The more of AI I see, the more I think it's 20% useful product, 80% hype. And 20% maybe overly generous.

I think it's good tool for producing ideas/concepts/topics. I don't see it becoming reliable source of information any time soon unless it actually understands what it's talking about and even then there are other issues that will arise with AI being able to understand things. 

[thevictor390]

30 minutes ago, Monkey Dust said:

AI can't understand its own output. This makes AI validating itself, and therefore producing reliable answers, somewhat challenging.

 

The more of AI I see, the more I think it's 20% useful product, 80% hype. And 20% maybe overly generous.

It's the current equiavalent of the .com bubble. Yes, the Internet changed the world. Yes, it was an incredible advancement. Yes, it made some people very rich. But it also produced a whole lot of hype-train-riding busts.

 

The same thing will happen to a lot of these "AI" technologies and startups. A few will win, most will lose, things will change.

[Sauron]

49 minutes ago, Eric the Grey said:

If AI models cannot even correctly tell us how long until the next US election day, or even report the same number of days when asked multiple times, they have a long way to go before they become trustworthy.  

They're large language models, not large correct information models... what did you expect? The only goal of these systems is to provide an answer that appears to come from a human. Correctness is completely incidental.

 

Also this is only going to get worse with recent news because of the flood of AI generated slop being fed back into the system...

[Dr. Will0hlep]

The thing with our current crop of AI chatbots is that their trained to have a conversation and/or write and article, ect. To do this they have analyised tens/hundreds of thousands of webpages to determine how conversations/articles/ect are normally structured and then they've had the AI attempt to learn to estimate/predict which word should come next, based only on the words that came before.

 

They have become very good at working out what words should come next, but that is all they are doing. Just choosing the next word based on the words that came before. It has no understanding of what it writes, nor any understanding of the individual words it selects. All it knows is which words are more likely to follow other words.

 

Given this, it was clear that these chatbots were never going to be able to provide always accurate information. Given the training data (human writing), it should have been clear that it was going to be wrong alot of the time.

 

Ever asked ChatGPT to do maths, or asked it to play chess...

[jaslion]

50 minutes ago, okkee said:

Is anybody trusting the info they get from AI. Like, it just gets a whole ton of things wrong and it seems like the companies are also trying to avoid privacy suits. in addition to that, it's neutered for some reason and will flatout refuse to output answers for certain topics. like, just dont use it for info.

Basically of thr hundreds of people I work with maybe a dozen mistrust it? The rest see it as fact or mostly factual which is good enough for them.

[Biohazard777]

13 minutes ago, Eric the Grey said:

~3:07: Guest Brian Sokas (Co-Founder and CTO @ GroundTruthAI), says:
"... the decisions they are making in the ballout box aren't quite informed by true facts,
they are just informed by information that they think are true facts"
Said by a dude with "Master's Degree, Engineering, Computer Science @ University of Pennsylvania"

 

52 minutes ago, Eric the Grey said:

My thoughts

If AI models cannot even correctly tell us how long until the next US election day, or even report the same number of days when asked multiple times, they have a long way to go before they become trustworthy.  

My toaster sucks at drying laundry.
Is the appliance bad, or the advertising for it which included ridiculous claims that it can do a bunch of other stuff as well?

[Dr. Will0hlep]

39 minutes ago, jaslion said:

Basically of thr hundreds of people I work with maybe a dozen mistrust it? The rest see it as fact or mostly factual which is good enough for them.

Actually terrifying, but my own experiance says your right.

[RejZoR]

I only find whole Ai thing useful if I'm looking for something highly specific and I don't want to dig through many search results if I'm in a rush to get an answer. Though most "Ai" are super picky and refuse to answer product details or straight up claim the product wasn't released yet even though it was and stuff like that. And for certain conversions and calculations where I don't want to dig around webpages and I want it to just do whatever for me where search engines would just point me to resources. I still prefer regular search when I have time to dig through hings and figure it out by myself. Seems more reliable.

 

Funny thing is, if Ai was so great and borderline godlike, shouldn't it be able to cross reference many resources and establish on its own what's factual and what isn't instead of just churning out absolute horseshit and having asterisk of "it might be hallucinating" and user can't really ever know when the Ai is on drugs? It's funny.

 

Btw, DuckDuckGo just added LLama and Mistral to their chat thingie (which already had ChatGPT and Claude) which is pretty cool since it runs in browser and restricts training of models with your queries. You can access it directly at www.duck.ai

[Kilrah]

-> Moved to General Discussion

 

If we had a dedicated thread about every way LLMs can be wrong we'd be drowning in... I prefer not to know what.

[Erioch]

9 minutes ago, Kilrah said:

If we had a dedicated thread about every way LLMs are wrong we'd be drowning in... I prefer not to know what.

I've been eating a pebble everyday and mixing glue with my pizza sauce and everything has been okay so far!

[Lurking]

4 hours ago, Eric the Grey said:

Summary

Study shows that more than a quarter of election information shared by AI factually incorrect

 

Sources

 

I bet half of the election information  shared by humans is factually incorrect. So AI already is an improvement.

 

And I don't know if you meant it sarcastic, but using MSNBC as a "source" about lies is a bit funny. MSNBC and Fox are heavy in the "skirting the truth" business. 

[LAwLz]

The one thing I hate more than companies using "AI" as a buzzword is people who use it incorrectly and then believe that is a valid argument to dismiss the technology completely.

 

Chatbots are not bad. They are incredibly valuable tools when used appropriately. Criticizing them for not being able to perform tasks they weren't designed for is like criticizing a hammer for not being able to saw wood. It's all about understanding their strengths and using them accordingly.

 

People need to stop with this child-like black-or-white way of thinking. You don't see people calling forks "stupid and useless" just because they don't work well to eat soup with. 

 

Use the right tool for the right job. ChatBots and other AI tools are fantastic at some tasks, but if you try and use them for everything then you will be disappointed a lot of the time. That's not the ChatBots fault. That's your fault for not using it for the right thing. Just because you CAN try wand eat soup with a fork doesn't mean it is a good idea. Just because you CAN ask a ChatBot how many days there are left until the election doesn't mean it is a good idea.

 

My rule of thumb is that if some question has a definitive answer, you shouldn't type it into a ChatBot. You should type it into Google. ChatBots are far better for questions that do not have definitive answers, or requires a lot of context that is difficult to include in a search query. 

 

 

Also, I am very surprised that ChatGPT-4o only got 17,5% of the answers incorrect. That is most likely far better than the average person.