Siri Gives Eagles 33 False Super Bowl Wins in Basic Knowledge Test

[Preddan]

This is interesting but Siri has always been a basic personal assistant and not a source of general knowledge. The steps towards being more of a generalist tool are just baby steps.

These results are bad but Siri also wasn’t designed to do things like this. It’s like asking your executive assistant to perform heart surgery. Without the proper training, you’re just asking for failure.

Siri clearly needs to improve but it’s also important to recognize that this test is asking Siri to do things it wasn’t trained to do.

Your comment is baffling. Unless your executive assistant is a psychopath with an urge to kill, surely he/she will inform you it is outside his/her skillset. Not just go ahead and wing it, like Siri did in this case and made up answers out of thin air. Why didn't Siri just say it didn't know the answer because it wasn't trained on that line of questions?

Also, as several has mentioned - Siri was trained on sport knowledge.

 

[neuropsychguy]

The Speakable Items that were part of MacOS classic and early OS X were more capable than this abomination.

As someone who used those old tools extensively back in the 1990s and 2000s, I can say they were absolutely not more capable than Siri.

 

[neuropsychguy]

Why didn't Siri just say it didn't know the answer because it wasn't trained on that line of questions?

Also, as several has mentioned - Siri was trained on sport knowledge.

My comment was about Siri before the updates. The old Siri did not answer the question. That’s covered in the article: "Gruber found that old Siri (i.e. before Apple Intelligence) did a better job at answering a question by declining to answer it, instead providing a list of web links." The updated Siri is taking steps towards being a generalist tool and so is now confabulating (“hallucinating”) but the old Siri would not answer.

Siri (old Siri) was not trained on general sports knowledge. If it was, it would have answered the questions. It was about current/recent events and people. You can test it. Get a device with the older Siri and ask it all sorts of random general questions to see how it does and what it says. It will almost certainly refuse to answer and offer to search the web for many/most. That means it isn’t trained to do those things and answer those questions.

The issue is with the new Siri that is confabulating ("hallucinating") answers that are grossly incorrect. They sound like GPT 3.5 or 4.0 type of answers without the ability to access the web and integrate web results into answers.

I’m not defending the updated Siri but am clarifying what the old Siri was not. The new Siri clearly needs to be much better.

 

[brofkand]

The next step is of course to change it because they have nothing else to do and ruin it (photos app anyone?)

If Apple and Microsoft were smart, they would have held the line with Snow Leopard and Windows 7. There was no need to make major changes after that.

 

[mjschabow]

You’re asking it wrong.

 

[Cunir]

Half the time when I ask my HomePod to play a particular song it will offer to do a “web search for that” on my phone

 

[Apple Knowledge Navigator]

I’m honestly baffled that the company which pioneered Spotlight on the Mac could get Siri so wrong.

I’ve turned off the majority of AI and feel like I’ve got my Mac back.

 

[spartan1967]

If I have other reliable objective resources available, I will always take beauty over brains.

 

[CarAnalogy]

Siri has been and always will be useless for anything other than simple things like setting timers. Apple has not written good software in years. The fact that their platforms are still mostly usable speaks to how far ahead they were a decade+ ago.

The really ironic thing is that Cue is apparently a sports fan and has specifically instructed the Siri team to make it good with sports information. They advertise Siri during NFL games.

 

[downpour]

Apple's text message summary has scared the hell out of me by completely changing the meaning of texts I've received. Apple's AI is not fit for purpose and they should shut it down.

 

[il_teo77]

All the Apple Intelligence coating will not hide the fact that Siri needs to be demolished and rebuilt from the ground up instead of being a patchwork. Apple needs a "snow Leopard" moment in iOS so much. Stop adding stuff. Take 12 months to polish, slim down iOS and redo all the plumbing instead. But I guess only a Steve Jobs would have the ability to sell that as a revolution to the general public. Nowadays Apple is so big and so iPhone dependent that I imagine shareholders and customers would be really vocal if Apple ever decided to take a sabbatical from adding stuff to iOS.

On the other hand, proclaiming features at WWDC and then taking a year to implement them hurts iPhone sales so I still have hope.

There's so much that needs to be redone "behind the scenes".

 

[GeoStructural]

So, iOS and macOS Zucks.

Those are more or less the best and most stable Operating systems.

iOS is stable, but I would not say it is better than barebones Android (no skin) feature-wise.

MacOS is also stable, but not necessarily better than Windows in many ways. There are things that are easier in Windows, and for fields like Civil Engineering, Mechanical Engineering, Computational Fluid Dynamics Windows is superior, more compatible with workflow and most software won’t run on MacOS unless you virtualize it and it becomes a slug fest. Gaming is also better on Windows.

 

[NMBob]

Maybe there's been a glitch in the Matrix making Siri right, and us wrong.

 

[drewmt24]

its misinterpreting super bowl LX as 1960 when the eagles beat the packers in the NFL Championship (prior to super bowls). Just another example of ai mixing up things. An easy fix. Go Birds!

 

[Kramerjn]

I asked, “Who won Super Bowl one”
It came up with Super Bowl Li, Patriots.

I asked, “Who won the first Super Bowl?”
It correct said Super Bowl l, Packers.

Weird

 

[ericinboston]

Forget Siri and forget Apple AI...Apple can't even get Autocorrect to be REMOTELY helpful when texting. Heck, it won't even capitalize common first names or, gasp, be smart enough that when I'm texting Wendy and I say "Hi wendy" it refuses to capitalize her name (again, even though she's the Contact).

Such terrible technology from a company that touts creativity and innovation. I guess Apple really does "think different"

 

[CopyChief]

"Plausibly wrong" is, perhaps, the best description I have ever read of 99% of the genAI responses I have seen, from Siri, from GPT, from Claude, from anywhere.

 

[Rradcircless]

*unsurprised chuckle*

 

[cjsuk]

If Apple and Microsoft were smart, they would have held the line with Snow Leopard and Windows 7. There was no need to make major changes after that.

I'm glad they cleaned up some of the design issues but yeah I know what you mean there.

 

[Onelifenofear]

American Handball.... who cares?

 

[Kramerjn]

You’re asking it wrong.

You say it as a joke but…

I asked, “Who won Super Bowl one”
It came up with Super Bowl Li, Patriots.

I asked, “Who won the first Super Bowl?”
It correct said Super Bowl l, Packers.

 

[dasmb]

Maybe Siri just isn't into sports...

 

[DEMinSoCAL]

Welcome to the walled garden, where you only get Apple's pre-approved, biased and politically correct version of the truth.

Based on the language of your post, I'm guessing you don't own a mirror?

 

[Razorpit]

American Handball.... who cares?

Well at least it's more exciting than European football. 😁

 

[DEMinSoCAL]

Well timers, and half the time Siri also manages to unlock my front door. Other than that, it’s nearly useless.

I about only use Siri for timers and half the time my request is ignored.