Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Siri Gives Eagles 33 False Super Bowl Wins in Basic Knowledge Test
- Thread starter MacRumors
- Start date
- Sort by reaction score
Siri is just absolutely the worst. It hasn't got smarter at all in 10 years as far as I can tell. I don't know what that VP of Artificial Intelligence is doing but he should be fired if he hasn't been already. Apple Intelligence is just the same old stupid Siri plastered in more obnoxious ways throughout the UI. TERRIBLE
Apparently just making things thinner with an extra hour battery life and more speed than anyone needs on a phone doesn't make Siri more useful.
Last edited:
Indeed, Siri is utter garbage. I mean, these were relatively basic questions, and Siri utterly failed. After pouring in billions of dollars in AI, and Siri cannot get some basic facts straight that takes you 1 minute to Wiki. What is the use? Same for the "summarizing" notification: I would immediately turn those off.
Note that Siri can only set multiple timers since iOS17....This has got to be laziest and crapiest "assistant" ever. Even the 90s paperclip in Word was more useful. But new Siri has such a nice new fancy glow when you activate it.
I feel that the mainstream AI "revolution" is starting to fall apart and has not advanced by a lot. Not to say AI has a lot of serious use cases (protein engineering (e.g. AlphaFold), help with coding etc.). But those latter use cases utilize (very) high quality input data for training AI models, not the utter drab that is spewed on internet.
Offtopic: In the meantime, Apple, please utilize your immense work force to fix Fotos. Why no 1-5 star rating, tagging, better raw support, panorama stitching? Those would not cost billions of dollars to fix. And you would make way more people happy than some Siri that cannot gets facts straight.
Offtopic 2: I think Apple has become lazy from drowning in money: Apple Car failure with billions down the drain and it does not even tip the balance of their profits. Now this.
With the results like this on Siri, I would never have let her be in charge of a car that I was in.
And given how it has gone with Siri for over a decade, I have no faith at all that they will ever catch up (let alone lead) in this area
imo the software has not followed pace with the hardware. some small cool stuff here and there, but its just a constant annual release of nothing spectacular
This is what happens when spend all week working on the vision pro, then realize the homework assignment was actually chapter 3, so you rush to complete it 10 minutes before class.
You're correct. I asked the new Siri, "Who won the 8th Super Bowl?" Here is the answer (see attached)
"Miami Dolphins", which is correct. However, the person who did the "methodological experiment" got an incorrect answer: https://onefoottsunami.com/2025/01/23/not-so-super-apple/#fn1-2025-01-not-so-super
Sounds like there are some methodological problems. So how Siri is asked influences the results. "Who won the 8th Super Bowl" will give a correct answer but "Who won Super Bowl eight" will not (but "who won Super Bowl 8" will give the correct answer).
Edit: And if you ask "Who won Super Bowl xiii" it will give an incorrect answer ("Pittsburgh Steelers"), which means it doesn't do Roman numerals well. Of course, there's a reason the world doesn't really use Roman numerals much...
Attachments
This is stuff that in a normal universe would and should keep a product "in the lab" for more work
The public should not be having to deal with something that is continually botching answers it should nail, correctly, ALWAYS
It's totally nuking their reputation and consumer trust -- which is really hard to get back
Today. Driving. 6:30 am. I activate Siri on Carplay via the steering wheel button.
Me: What's the temperature?
Siri: It is 27 degrees. Temperature is heading down from 27 degrees to 18 degrees tonight.
Immediately after
Me: What is today's predicted high temperature?
Siri: Today's high will be 43 degrees.
The response has been like the first one for years. Clearly it implies that the temperature at 6:30 am will be the high for the day. The correct response would be:
"It is currently 27 degrees. Today's high will be 43 degrees and tonight's low will be 18 degrees".
Or, less informative but still correct:
"It is 27 degrees".
Me: What's the temperature?
Siri: It is 27 degrees. Temperature is heading down from 27 degrees to 18 degrees tonight.
Immediately after
Me: What is today's predicted high temperature?
Siri: Today's high will be 43 degrees.
The response has been like the first one for years. Clearly it implies that the temperature at 6:30 am will be the high for the day. The correct response would be:
"It is currently 27 degrees. Today's high will be 43 degrees and tonight's low will be 18 degrees".
Or, less informative but still correct:
"It is 27 degrees".
A good personal assistant is key element for future growth. Everybody has on, except Apple.
Timmy is more interested in grabbing as much as he can from its products, but forgets he has in to invest in them too to stay relevant in the future.
I think he will probably retire next year. Leaving behind a financial institution named Apple with no vision for the future and scrambling behind everyone else.
Pathetic. No wonder people are not interested in Apple Intelligence. Not only is it the biggest nothingburger Apple has ever served us, it can’t even get the simplest functions right! Have you noticed that the Apple weather app can’t even get the weather right, either? It’s ludicrous. Something is seriously wrong in Cupertino. Innovation lost years ago, basic functions now failing. I think it’s time for new management. Across the board.
In what may not come as much of a surprise, a new test of Siri's knowledge of Super Bowl history has revealed significant accuracy issues with Apple's virtual assistant, suggesting Apple still has some way to go in overcoming challenges with Siri's ability to provide reliable information.
In a methodical experiment, One Foot Tsunami's Paul Kafasis asked Siri who won each Super Bowl from I through LX and documented its responses. The results were strikingly poor, with Siri correctly identifying winners only 34% of the time – just 20 correct answers out of 58 played Super Bowls.
Perhaps most notably, Siri repeatedly and incorrectly credited the Philadelphia Eagles with 33 Super Bowl victories, despite the team having won only one championship in their history. The virtual assistant's responses ranged from providing information about wrong Super Bowls to offering completely unrelated football facts.
While Siri did manage a few streaks of accurate answers, including three consecutive correct responses for Super Bowls V through VII, it also had a remarkable string of 15 consecutive incorrect answers spanning Super Bowls XVII through XXXII.
In one telling instance, when asked about Super Bowl XVI, Siri offered to defer to ChatGPT - which then provided the correct answer. The contrast highlighted the limitations of Siri's own knowledge base compared to more advanced AI systems.
The test was conducted on iOS 18.2.1 with Apple Intelligence enabled, and similar results were found on both the upcoming iOS 18.3 beta and macOS 14.7.2, suggesting the issue extends across Apple's platforms. Kafasis generated a spreadsheet of the results in both Excel and PDF formats, which you can read here.
Separately, inspired by Kafasis' test, Daring Fireball's John Gruber tried some of his own sports queries with Siri and compared its responses to ChatGPT, Kagi, DuckDuckGo, and Google, all of which succeeded where Siri failed.
Perhaps worse for Apple, Gruber found that old Siri (i.e. before Apple Intelligence) did a better job at answering a question by declining to answer it, instead providing a list of web links. The first web result provided an accurate, if only partial, answer to the question, whereas new Siri, powered by Apple Intelligence, fared much worse. Gruber explains:
"It's just incredible how stupid Siri is about a subject matter of such popularity," commented Gruber. "If you had guessed that Siri could get half the Super Bowls right, you lost, and it wasn't even that close."
Of course, this isn't the first time Siri has received heavy flak for its all-round performance, but Gruber's criticism about "plausibly wrong" answers to general knowledge questions ties back to the modern problem of hallucinating AI chatbots that spout misleading or flat-out wrong responses with complete confidence.
Apple is developing a much smarter version of Siri that utilizes advanced large language models, which should allow the personal assistant to better compete with chatbots like ChatGPT. A chatbot version of Siri would likely be able to hold ongoing conversations and provide the sort of help and insight as ChatGPT or Claude, but how well the integration will perform may be a concern, going on Siri's abysmal track record.
Apple is expected to announce LLM Siri as soon as 2025 at WWDC, but Apple won't launch it until several months after it's unveiled. That means LLM Siri would come in an update to iOS 19, with Apple planning for a spring 2026 launch.
Article Link: Siri Gives Eagles 33 False Super Bowl Wins in Basic Knowledge Test
Last edited:
Why would you use Siri for AI tasks that require accuracy? Siri is irrelevant in the world of AI except for elementary tasks such as asking it the temperature, time, HomeKit integration etc… and works about 50% of time. Basically Siri is a mediocre personal assistant that integrates with iOS to perform tasks using AI technologies.
Use Open AI’s ChatGPT or Google’s Gemini. They’re not perfect but far more capable than Siri.
Use Open AI’s ChatGPT or Google’s Gemini. They’re not perfect but far more capable than Siri.
It's because Tim turned the whole place into a financial engineering operation that also sells some electronics
It would truly be sad if either ChatGPT or Gemini were less perfect than current Siri seeing as they have tens of billions in deployed LLM server centers backing those solutions versus your phone and a thin client.
Basically, to sum up your off topic:
Please, for the love of your customers, divert some resources from AI and bring Aperture back!!! (Face recognition was introduced in Aperture and it performed more or less as it does today, which says a lot about the whole global AI (not only Apple’s) kerfuffle. Apple should have continued to call it machine learning. Calling it AI for shareholder’s sake was the worst strategic choice Apple made in 2024, and then some.
Am I the only one who has gotten incredibly bored by the “look how bad Siri is hahaha” posts. Is this was counts for news now days? Who is relying on a voice assistant for anything critical? They get things wrong, nothing new here.
These are increasingly important interaction methods and Apple's is terrible
It's only going to get more and more newsworthy to discuss its shortcomings
I’m really wondering what’s going on at Apple. Siri still can’t handle the task from apples own promotional video’s from 14!!!! years ago.
With the introduction of Apple Intelligence the shouted out big how they would integrated the AI-related things into iOS 18. The most AI-related things still aren’t available and the things that could be useful were generating fake news.
In the meantime Samsung released their new OS One UI 7 that does all the things Apple promised for iOS 18. Same look and feel (and a button to close all opened apps at once), a working smart personal assistant that’s able to handle follow up questions and turning them into useful working actions. All the things Apple was heavily promoting with iOS 18.
Apple might launch new MacBook airs and iPad 11 in march. Probably only a chip swap and the much needed ram. Still no working Siri and buggy iOS.
A working Siri might be available, not from the start, in iOS 19 in 2026. That’s more than a year from now!
So Samsung is basically 2 years ahead of Apple on software. By the time Apple is ready what One UI 7 can do, One UI 9 is out of the gates. I don’t see Apple being able to catch up.
A total embarrassment for such a rich company. Fire their ceo!
With the introduction of Apple Intelligence the shouted out big how they would integrated the AI-related things into iOS 18. The most AI-related things still aren’t available and the things that could be useful were generating fake news.
In the meantime Samsung released their new OS One UI 7 that does all the things Apple promised for iOS 18. Same look and feel (and a button to close all opened apps at once), a working smart personal assistant that’s able to handle follow up questions and turning them into useful working actions. All the things Apple was heavily promoting with iOS 18.
Apple might launch new MacBook airs and iPad 11 in march. Probably only a chip swap and the much needed ram. Still no working Siri and buggy iOS.
A working Siri might be available, not from the start, in iOS 19 in 2026. That’s more than a year from now!
So Samsung is basically 2 years ahead of Apple on software. By the time Apple is ready what One UI 7 can do, One UI 9 is out of the gates. I don’t see Apple being able to catch up.
A total embarrassment for such a rich company. Fire their ceo!
I’m getting totally bored of a still not working Siri after 14 years. I’m getting bored of the buggy software releases and getting bored of the hardware that is the same year after year after year with only chip swaps and a color change.
I’m also getting bored of people defending this and not seeing that Apple is getting behind on all fronts.
It’s a total embarrassment for the software engineers at Apple and their CEO’s to completely miss the boat.