How can AI ever be accurate?

[antonypg]

I have been playing around with ChatGPT recently, trying to use it as a search engine instead of Google or Duck Duck Go.

When planing weekends away, holidays etc. it can give quite informative results, in general terms, but it does come up with lots of little mistakes.

Someone on the Disney Forums recently complained that the parks were closing early on the dates of their holiday. The parks actually weren't closing early, but they had asked ChatGPT rather than checking on the official website.

Recently I asked ChatGPT to generate an image of an Apple Studio Display and an M4 Mac mini. It took quite a while, then gave me a very good looking image of an Apple Studio Display and an earlier Mac mini. When I asked it what Mac mini was in the picture, it stated it was probably an M2 or M4, but most like an M4!!

When I asked what ports the M4 Mac mini had, it gave me the correct information, but when I asked it to produce a diagram it was completely wrong.

I don't understand how these AIs can ever be trusted if they just scour the internet looking for information. Surely most of the information available on the internet is inaccurate or just complete lies. These engines need to be seeded with just 100% correct information, which is an impossible task.

 

[arc of the universe]

you touch on an interesting point.
what is "correct".

think of a time before the internet.
there were written materials (the Bible's denominational flavors, encyclopedias, almanacs, dictionaries) where people looked to find authoritative information in these sources. people generally had shared values supported by these printed sources.
so, in pre-internet: knowledge=fact=written word=shared value. creation of published content was tightly controlled.
but here's the thing: the means to produce a written work was not mass oriented. people whose work was published were trusted originators.

now the internet.
anybody can blog or v-blog or audio podcast.
the internet is anonymous.
the internet is the modern publishing vehicle.
and anybody and everybody says whatever they want.
clicks=truth. think InfoWars.

now for me.
google AI generally gives me information and answers that i can cross check with sources that i believe are correct.
but i cross check AI information with sources like Mayo Clinic; New York Times; Wikipedia (!), etc.

its not hopeless. but difficult.
it turns out that AI engines are very much like people.
they lie; they scheme; they become lazy.
and sometimes they are brilliant .

for persons spending a lot of time on Youtube and TikTok and Facebook, and Instagram and Truth Social and Fox and X, they will never get out of their unending cycle of conspiratorial and populist world of generated "alternative facts".
AI is simply reporting snippets of information that is on the internet. if many people repeat: "a room at the intercontinental Paris costs only 100 USD this Saturday" the level of AI that we have right now is simply that it regurgitates this junk info.

the reality of a divided, conspiratorial, and anti-science world, and, growing inability to fact check, will make our life much more difficult to determine what is actually true.

google AI, in the recent past, gave its summary, then gave you a list of websites you can go to review the places it is using. however, many times more recently, it doesn't give this list.

 

[JSRinUK]

Chatting with AI (or LLMs) is a process known as “inference”.

inference /ĭn′fər-əns/

noun​

1. The act or process of deriving logical conclusions from premises known or assumed to be true.
2. The act of reasoning from factual knowledge or evidence.
3. Something inferred.
4. A hint or suggestion.
   "The editorial contained an inference of foul play in the awarding of the contract."
5. The act or process of inferring by deduction or induction.
6. That which inferred; a truth or proposition drawn from another which is admitted or supposed to be true; a conclusion; a deduction.

The LLM is inferring based on predicting the most likely word to follow the previous one, based on its training data.

If most people in the world have claimed that 2+2 = 5 (even if they all said “2+2=5 is clearly wrong”) then ChatGPT (and others) will have sucked this up and it will tell you with confidence that 2+2=5 because it determines that “5” most often comes after “2+2”. It’s inferred a truth based on a pattern that’s fundamentally wrong.

 

[whatnot]

I don't understand how these AIs can ever be trusted if they just scour the internet looking for information. Surely most of the information available on the internet is inaccurate or just complete lies.

think Youtube. is it Internet? yes. does it contain 'accurate' (as in description matches content) information? yes. can you find it using Youtube search even if you enter it verbatim? never.

Google's first page is a heap of unrelated trash with 'AI' summary of said trash at the top, which is presumably then scanned by AI in search of answers, and so on.

leaving 'creative' options aside, 'AI' is just an interactive spreadsheet throwing up whatever it's been fed, so why would anyone rely on it is anyone's guess.

 

[antonypg]

It is really hard to say what "AI" means. There is no intelligence, just software algorithms doing pattern matching, but I guess the answers/responses given somehow infer that there is some sort of intelligence.

 

[Just_Kevin]

This closely matches my opinion and use case for AI as it exists currently. Garbage in = garbage out.

I find it useful for creating comparisons between products, doing price checks and availability and things like that. I have never fully trusted any one source of information, I put a higher level of skepticism on AI responses. I also always have it cite the sources.

This topic came up at work the other day. One of my students and his friend were able to get an AI to believe that a gun manufacturer had a job opening for taste testers. The AI even helped them create a resume and practice for a hiring interview for the job.