As far as this forum is concerned, if Siri doesn't come last they're not testing it right.
Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Siri Answers 83% of Questions Correctly in Test, Beating Alexa But Trailing Google Assistant
- Thread starter MacRumors
- Start date
- Sort by reaction score
Apple appears to be fairly good at making money and making sure Siri is a solid voice assistant is a key piece of their ecosystem and continued revenue. The difference is that Siri sells hardware and not info about you like Alexa and Google. So I agree it is an important consideration when deciding on a voice assistant.
[doublepost=1565916501][/doublepost]
If we are going to play the my experience game, give specific examples so we all understand. In my experience Siri extracts the actual question out of a statement better than Alexa and normally gives a better response as the study suggests. But it is a bit dependent on what you are asking for. Siri doesn't buy things for you like Alexa. So what examples do you have.
Why isn't Cortana on this list?
Actually, Loup Ventures did test Cortana last year and as usual it came dead last:
"Google Assistant continued its outperformance, answering 86% correctly and understanding all 800 questions. Siri was close behind, correctly answering 79% and only misunderstanding 11 questions. Alexa correctly answered 61% and misunderstood 13. Cortana was the laggard, correctly answering just 52% and misunderstanding 19."
Siri works beautifully in my own experience.
[doublepost=1565917004][/doublepost]
Except that in 2017, Loup Ventures (and Gene Munster) put Siri dead last (even worse than Cortana) in that year's Voice Assistant study:
"Three HomePods were subjected to 782 queries by the firm, said analyst Gene Munster. While Siri understood 99.4 percent of them, it was only able to answer 52.3 percent of them correctly. The latter figure compares with rates of 81 percent for Google, 64 percent for Alexa, and 57 percent for Cortana."
So Gene doesn't let being a "heavy bull on Apple" get in the way of his objectivity.
Apple should've paid more for this fake research so it can surpass Google.
Still rely on Reddit for real results.
https://www.reddit.com/r/sirifail
Still rely on Reddit for real results.
https://www.reddit.com/r/sirifail
Better check out Alexa fails and Google fails as well for completeness though:
https://www.reddit.com/search/?q=Alexa fails
https://www.reddit.com/search/?q=Google Assistant fails
I’ve just never had those particular issues. That’s not to say I’ve never had an issue of any kind but that I’ve not had anything like what you’re describing be an issue for me.
Sorry to be so harsh, folks, Siri is terrible. Dead last. A librarian in a small town is right more often.
At this point I only use Siri to set timers and adjust my HomeKit thermostat. Even for those basic, repetitive tasks, its success rate is about 55-60%.
Would be fascinated to know to what extent the “success” metrics have been heavily swayed by selection bias in two ways:
1) Most people I know long ago gave up on Siri, so their invalid requests are no longer being counted as unsuccessful by Apple
2) Of those who do use Siri, they resort to only the most basic requests. Clearly, its success rate will go up when you teach people for years that all but a very small subset of requests have any chance of being processed.
Would be fascinated to know to what extent the “success” metrics have been heavily swayed by selection bias in two ways:
1) Most people I know long ago gave up on Siri, so their invalid requests are no longer being counted as unsuccessful by Apple
2) Of those who do use Siri, they resort to only the most basic requests. Clearly, its success rate will go up when you teach people for years that all but a very small subset of requests have any chance of being processed.
Then, you are buying for the wrong reason. Maybe you are buying just for a logo? Or because all you friends have it.
[doublepost=1565922326][/doublepost]Hmmm! I just tried it and, apart from the commerce area, Siri did great; though I note I did not use 'Uptown' because its a stupid phrase for most people. Instead, I asked how to get to the city centre. No fuss. No bother for Siri.
I wonder how much its tied to people's enunciation though. I do note that my brother in-law, who mumbles and slurs words, does have issues with Siri - less so with Google, but still not great.
Last edited:
Yes exactly! I’m ready to throw the HomePod out the window! My phone picks up the request instead of the HomePod. I know for a fact Siri’s listening skills have become worse and worse. It’s very frustrating.
Hello Google...
It will take you 35 mins today to drive to your work but you have a meeting at 8 so you should leave now and pick up a double shot latte for your boss who I know did not make a stop at the coffee shop as they have 326 times this year. I have checked your entire driving history and find you drive 6% slower so leave sooner. By the way would you like to buy a new coffee machine from best buys, its on special this week
Hey Alexa
Yes? By the way I heard you arguing with your wife about that new car, I have a book you can buy about spousal disagreements, if you would prefer to listen to it, say install Audiable, or if you want to watch it, I can sign you up for Amazon prime, would you like to check out now?
Hey Siri
Yes?
It will take you 35 mins today to drive to your work but you have a meeting at 8 so you should leave now and pick up a double shot latte for your boss who I know did not make a stop at the coffee shop as they have 326 times this year. I have checked your entire driving history and find you drive 6% slower so leave sooner. By the way would you like to buy a new coffee machine from best buys, its on special this week
Hey Alexa
Yes? By the way I heard you arguing with your wife about that new car, I have a book you can buy about spousal disagreements, if you would prefer to listen to it, say install Audiable, or if you want to watch it, I can sign you up for Amazon prime, would you like to check out now?
Hey Siri
Yes?
This analysis seems consistent with my experience. I currently have 2 HomePods and I am down to 1 Amazon Echo which I plan to replace soon with a 3rd HomePod. I only use Alexa and Siri for "Information" and "Voice Commands" with my smart lights and smart outlets. Hands down, Alexa beats Siri with "Information" bus Siri is MUCH better (faster and better consistency) at controlling smart home devices which is my main use of Siri. I am looking forward to the new capability iOS 13 will bring with voice activated shortcuts!
I did these with Siri across the room from me with Hey Siri on an iPhone 6s.
My replies:
- The nearest coffee shop is [Shop Name] Cafe. Is that the one you're looking for?
- I couldn't find any matching places.
- One option I found is [Place name]. Is that the one you're looking for? (Yes) Getting transit directions to [Place name]
- The twins definitively beat the Rangers by a score of 13 to 6 today
- Ok, your remind is set (Shows the reminder to call Jerome)
Either you're in an extremely loud environment or you're using a much older iOS version, I can't see how you got these weird incorrect replies. My Siri pretty much got what I said word for word.
[doublepost=1565928178][/doublepost]
Maybe get your mic checked at an Apple Store? My Apple Watch is usually just as good as my iPhone at picking up what I said.
I personally use Siri for far more than basic tasks.
I use Siri on the HomePod, Apple Watch, Apple TV and iPhone to control and monitor my entire house including the backyard Pool pump and pool heating, the robot vacuum cleaner, all the lights, the security cameras, multiple smart switches, 2 Apple TVs, Home Security System and multiple Macs. The only thing I can't control so far is my ducted Air conditioning, but that doesn't work with Alexa or Google either. And I have the reassurance that my home is far more secure and private than if it was controlled using Amazon or Google.
I ask Siri what the temperature is in the living room, to convert ounces to grams, to place and receive phone calls, to play "music I would like in the Living room and kitchen", to add items to the shopping list, to start the robot vacuum cleaning the house, to turn off or dim lights when I'm watching movies, Speech to text dictation on the iPhone and Mac and the list goes on.
It works extremely well with excellent accuracy in my experience with my Aussie accent.
Last edited:
Ok that makes sense. Objectively, Siri understands Gene Munster better after all these years.
Actually, you couldn't be more wrong. Siri is used many billions of times a week, and this study closely matches my experience with Siri and most studies that show she is ahead of Alexa and closing in on Google. I use Siri throughout the day, and she's consistently good.
More important. How does this compare to a human assistant? Would be intersting to see some controlled study where the complexity of the questions increased. The listed has no fun aspects of AI to compete with a human assistant.
By the way, 800 questions from one person-near meaningless study. Moreover, description of the study design seem to be lacking to allow the reader to judge confounding factors such as: where the questions randomised? Where each device tested after each other with the risk of bias? Where the questions placed in studio environment or in real life situations? Where the questions recorded and replayed to ensure equal volume and ambient noise?
By the way, 800 questions from one person-near meaningless study. Moreover, description of the study design seem to be lacking to allow the reader to judge confounding factors such as: where the questions randomised? Where each device tested after each other with the risk of bias? Where the questions placed in studio environment or in real life situations? Where the questions recorded and replayed to ensure equal volume and ambient noise?
Using Siri for test tonight it is as dumb as a rock. Keeps giving me apple centric info and links to apple apps and other BS. Google understands of course.
[doublepost=1565936855][/doublepost]
Crowd sourcing. Wisdom of the crowds.
Or as many analysts have pointed out, Apple has been improving Siri at a faster rate than other voice assistants, but feel free to believe whatever supports your worldview.
Indeed, Voicebot.ai reports that Siri dominates the Voice Assistant market in terms of actual usage to the tune of 45.6% marketshare compared to Google Assistant on 28.7%, Alexa on 13.2%, Samsung's Bixby on 6.2% and Microsoft's Cortana on 4.9%.
So it looks like Google Assistant and Alexa (and the rest) are the assistants that people stop using or don't use in the first place - not Siri.