Siri on iOS Answered 78.5% of Queries Correctly in Latest Test, Trailing Google Assistant at 85.5%

[DNichter]

That's not very eco friendly. Why didn't you recycle them or perhaps donate them to a charity?
[doublepost=1532526501][/doublepost]

I hope I never encounter you on the road whilst I am driving given you're so distracted.

I offered them to friends and family, but couldn't get any takers. They did go in my recycle bin actually, if that makes you feel better.

 

[Stella]

Ok. Have a nice day.

Thanks, I will.

.

Its sad, despite having Siri, which was way better than anything else at the time, Apple chose to rest on its laurels and let the competition easily catch up. Now, Apple is constantly playing catch up with Siri. I very much doubt Apple will catch up with Google for a long time yet.

 

[nelsonammo]

It's improving pretty fast.

 

[ThunderSkunk]

My grandfathers Jitterbug fliphone comprehends and answers 100% of all questions asked of it when you push the big red personal assistance button.

And it employs a staff of friendly helpful humans who are happy to get you whatever you need when you push it. I was joking when I started this, but it's actually pretty nice, now that I think about it.

 

[DNichter]

Thanks, I will.

.

Its sad, despite having Siri, which was way better than anything else at the time, Apple chose to rest on its laurels and let the competition easily catch up. Now, Apple is constantly playing catch up with Siri. I very much doubt Apple will catch up with Google for a long time yet.

They aren't that far off as is, as you can see by the tests. Google has years of customer data to help with this stuff, fortunately Apple does not. I think most are totally fine with Apple taking this approach. I am actually surprised Google isn't much further ahead at this point, considering the amount of data they have.

 

[EdT]

Compared to what? According to these results, Siri is a little or a lot better than both Alexa and Cortana. Siri is not quite as Good as Google Assistant but these tests are important because they invalidate all hyperbolic claims that Siri is worthless and/or Google Assistant is leaps and bounds ahead. We all have the feeling that Siri has fallen behind so these numbers seem to validate that. They do not validate opinions that claim Siri is either far ahead or way behind the competition.

I agree. This isn’t a test conducted by or expressly for Apple, it was done to compare functionality of all the major assistants.

I don’t know anything about the company testing, but frankly when I see any comparison article on test results I don’t really know how trustworthy or impartial the company really is. If this is horribly inaccurate then I’m sure another test will be done by someone refuting these results.

Personal results vary on even the best AI. Just because it doesn’t work for you, personally, doesn’t mean it’s not as good or better than other assistants.

I’m someone who has criticized Apple for wasting the lead that they had in voice assistants. If they are catching up to be one of the better ones then that is an improvement that I didn’t expect. I hope these results are verified by other independent tests.

 

[LordVic]

I offered them to friends and family, but couldn't get any takers. They did go in my recycle bin actually, if that makes you feel better.

as an FYI, residential recycling programs are not suited to handle electronic devices and putting them in your recycling bin likely ended up causing them to go directly into the landfill, along with whatever else was in your bin at the time.

Electronics should be properly recycled by taking them directly to an electronics recycler, who will strip the parts down, remove precious metals and then send the individual components to the proper places for them to be recycled or reused.


Electronics should never be tossed in your curbside pickup

 

[Stella]

Customer data ( I'm assuming you mean - 'mined' data ? ) does not help with queries.

Given Apple had such a head start, I'm surprised that Siri isn't leading the pack by a large margin!

There's a reason why I have Google Home instead of buying a HomePod - Apple just isn't good enough, in accuracy, capability and third party integration.

Of course, YMMV. Use what *you* personally find better.

They aren't that far off as is, as you can see by the tests. Google has years of customer data to help with this stuff, fortunately Apple does not. I think most are totally fine with Apple taking this approach. I am actually surprised Google isn't much further ahead at this point, considering the amount of data they have.

 

[pika2000]

And yet Siri still cannot effectively do voice to text without messing up 75% of the time.

Siri will never be able to have a better voice data set than Google. Google has been transcribing voice mails from google voice for almost a decade.

 

[macduke]

I would like to know if the 800 questions were the same between the two tests and if so, which questions were able to be answered now that were not possible previously? The thing about assistants like this is humans get used to these assistants not being able to complete certain tasks so we stop asking because we're adapting to the assistant. If the assistant becomes smarter then it needs to let us know.

 

[DNichter]

Customer data ( I'm assuming you mean - 'mined' data ? ) does not help with queries.

Given Apple had such a head start, I'm surprised that Siri isn't leading the pack by a large margin!

There's a reason why I have Google Home instead of buying a HomePod - Apple just isn't good enough, in accuracy, capability and third party integration.

Of course, YMMV. Use what *you* personally find better.

You don't think that the millions of google searches (and all of the other spots they collect user data) each day helps Google with an AI assistant?

 

[Kabeyun]

Internally, Apple struggled with Siri’s purpose: command implementer or comprehensive digital assistant. These data, particularly the first graph, support that they’ve done a better job with the former. I’d also argue that it’s the far simpler job so it should come as no surprise.

But when I ask Siri the best time to visit Yellowstone National Park and it tells me it’s never heard of such a place, my aneurysm acts up.

 

[iZac]

If I tried to push a button on my keyboard and it failed 21.5% of the time, I wouldn't use that keyboard.

 

[Jsameds]

Personal Data mining is irrelevant.

Uh, no. No it isn't.

It's very relevant.

 

[avatar77]

Why test Alexa on an iOS device and Google on a Pixel? The test should have tested each assistant on their "native" platform - Alexa on an Echo, Cortana on a Windows 10 system (since Cortana lacks a truly popular speaker), Siri on a HomePod, and Google Assistant on a Google Home.

BTW, Alexa on an Echo Show is a revelation. The screen adds a lot to the output.

 

[jacksmith21006]

Sorry but was driving with my wife last week and she was trying to use Siri and it was just awful. It messed up in a way that caused us both to laugh so hard I was barely able to get to the side of the road.

Picked up my Pixel and the Google Assistant got it on the first try. Siri is still horrible.
[doublepost=1532529349][/doublepost]

I call BS. I have never got anywhere near 70% accuracy. Mind you I am not an American living in the Bay Area.

Suspect it was gamed. My experience has been Siri is just awful.

 

[Jsameds]

Sorry but was driving with my wife last week and she was trying to use Siri and it was just awful. It messed up in a way that caused us both to laugh so hard I was barely able to get to the side of the road.

Picked up my Pixel and the Google Assistant got it on the first try. Siri is still horrible.

So:

Siri didn't work with Person A with ambient driving noise, but Google Assistant worked with person B without ambient driving noise.

Not exactly a fair test now, is it?

 

[Carlanga]

Certainly puts to rest all the claims of "no improvement" in Siri, as well as a sizable dent in the claim that Alexa is more accurate.

lol no this is a biased test towards Alexa and Cortana so as to favor apple/google. Alexa and Cortana were tested inside their iOS app instead of their standalone devices which work muchhh better than inside their competitions devices.

 

[Nozuka]

Siri almost always understands what i'm saying. But she rarely knows how to answer.

 

[prusiner]

Here in Spain, Siri is able to answer correctly about 1% queries.

You can talk Google in cockney and he still will be able to answer correctly 95%+ of the queries.

Seriously Siri is a real pain (here in Spain)

 

[dman22]

I'm curious as to how often Siri is "updated" - are there only improvements when iOS is updated or are there continual, incremental background updates -- in other words, the "machine learning." Is that happening all the time or only on OS updates?

It is both. Apple makes improvements to Siri via Updates but mostly it improves Siri in the background on Apple servers.

 

[Meñito]

Yes, yes, only in english, in other languages it does not exceed 30% of correct answers.

 

[69Mustang]

So:

Siri didn't work with Person A with ambient driving noise, but Google Assistant worked with person B without ambient driving noise.

Not exactly a fair test now, is it?

I know right. About as fair as judging Alexa and Cortana from an iPhone app instead of a native device.
If they weren't going to use native devices, Alexa and Cortana should not have even been in the evaluation.

 

[question fear]

My issue with Siri is that she is very weirdly unreliable. I once tried to call someone at a place called RMA and gave Siri the town and state, and she tried to call the Royal Military Academy of Brussels. Definitely not in NJ.

She also has a tendency to cut me off partway through dictating tasks, so I'll get reminders that just say "tonight", or "take out".

Someday I'll remember what the mystery "don't forget" to-do was for.

And yes, I could go with just typing out my reminder, but I usually remember things at random times, so saying "Hey Siri, remind me at x time to do abc" is faster. When Siri properly transcribes things. Or at least transliterates well enough for me to sort-of guess.

I'm also still a little salty about the time I needed directions to my mechanic's in a hurry, and she snottily informed me the mechanic only had 2 stars, was I sure this was the right place?

 

[H2SO4]

Compared to what? According to these results, Siri is a little or a lot better than both Alexa and Cortana. Siri is not quite as Good as Google Assistant but these tests are important because they invalidate all hyperbolic claims that Siri is worthless and/or Google Assistant is leaps and bounds ahead. We all have the feeling that Siri has fallen behind so these numbers seem to validate that. They do not validate opinions that claim Siri is either far ahead or way behind the competition.

There is more to it than just a simple we asked a question. Can't Google answer two Q's one after the other as it retains context?