Siri Shows Increasing Promise, But Accuracy Falls Short of Google Text Searches

[samcraig]

Siri, when I get home, remind me that Siri does more than just search and dictation.

Hmmm..

Making phone calls and sending text-messages are the most popular activities, according to the report, a sign that Siri usage is mirroring how people use their phones more generally. Roughly a third of 4S owners use Siri to place phone calls, send text messages, or look up information daily or almost daily.

Many other Siri services are getting little pickup, however. They include playing music and scheduling meetings. Thirty-two and 35% of 4S users, respectively, said they had never used Siri to perform those actions. Those categories also had some of the lowest percentage of users that did either daily or almost daily.

Email usage is pretty split, with 30% saying they have never used Siri to send an email while 26% say they use it to send email daily or almost daily.

Source: http://blogs.wsj.com/digits/2012/03/26/apple’s-siri-gains-traction-for-some-things/

 

[JohnDoe98]

On the accuracy side, Google's 86% rating is derived from comScore data showing that Google generates 1.14 search result pages per search, suggesting that roughly 86% of time Google presents the data the user is looking for within the first few results.

So this the point. Will you get information quicker using Siri or googling? So you've completely missed the point here.

Seeing how Siri can use Google's search engine if instructed to do so, the answer is you won't get it any faster using one or the other. The only difference is the speech recognition is a little less accurate than typed-input, but the result will be the same.

It's not comparing two things at once. It is comparing how accurately you can get information using both tools. That's just one thing.

Again what you fail to comprehend apparently is Siri can use Google as it's tool, so it isn't an either or case. One tool is subsumable under the other. So, to make this clear to you, Siri can use either Google or Wolfram Alpha, your choice. So depending on what kind of search you want to do, you would be best advised to use the engines that yields the best results.

That would be biased. Secondly, read again: Google generates 1.14 search result pages per search, suggesting that roughly 86% of time Google presents the data the user is looking for within the first few results. So if you ask Siri "how tall is the empire state building", you'll get a precise answer from Siri (hopefuly) and find that information on the first page of Google results (actually, if you google that you'll now get an answer too). So this is what they're comparing. So again, you're completely missing the point.

But again you fail to realize you can ask Siri to "Google how tall is the empire state building". You are not forced to only say "how tall is the empire state building". Just because you don't know how to use Siri effectively doesn't diminish it's capabilities.

And again, just go and read the article.

Again, try and understand what I'm saying and try again if you like.

 

[SleeplessChaos]

Fine, if you want to embarrass yourself...

On the accuracy side, Google's 86% rating is derived from comScore data showing that Google generates 1.14 search result pages per search, suggesting that roughly 86% of time Google presents the data the user is looking for within the first few results.

So this the point. Will you get information quicker using Siri or googling? So you've completely missed the point here.

It's not comparing two things at once. It is comparing how accurately you can get information using both tools. That's just one thing.

That would be biased. Secondly, read again: Google generates 1.14 search result pages per search, suggesting that roughly 86% of time Google presents the data the user is looking for within the first few results. So if you ask Siri "how tall is the empire state building", you'll get a precise answer from Siri (hopefuly) and find that information on the first page of Google results (actually, if you google that you'll now get an answer too). So this is what they're comparing. So again, you're completely missing the point.

And again, just go and read the article.

----------


Another person who missed the whole point of this research...

I'm wondering if we should check your reading comprehension guys...

I don't think he is missing the point of the experiment, I think that he, like me, thinks the point is stupid.

Think about it for more than a millisecond and you'll realize that it's comparing apples and oranges. Google, like every other text based search engine on the web, is simply matching text keywords with a large indexed database built by years of crawling through the web. The search is basically brute force and simple.

Siri is receiving a digital waveform from the microphone, eliminating frequencies from that waveform that are not commonly found in the human voice, translating those remaining frequencies into readable text via a remote server and then interpreting that into a command which it then uses to either search the web using what it assumes to be the correct web search service or perform a function using a local app. That service list includes google search.

The fact of the matter is that the two services are not comparable whatsoever and that for all intents and purposes siri expands upon the abilities of google regardless of the situation.

 

[Dr McKay]

Gave up on Siri towards the end, I don't have a regional accent, or a speech impediment and it routinely got most of the things I said wrong. The "beta" excuse has long run dry, you don't release something and advertise the heck out of it as an amazing core feature, and then hide behind the Beta tag for nearly a year.

 

[SandboxGeneral]

Fascinating. Were the tests done with iOS 5.1.1 or iOS 6?

 

[zeromeus]

Fine, if you want to embarrass yourself...

On the accuracy side, Google's 86% rating is derived from comScore data showing that Google generates 1.14 search result pages per search, suggesting that roughly 86% of time Google presents the data the user is looking for within the first few results.

So this the point. Will you get information quicker using Siri or googling? So you've completely missed the point here.

It's not comparing two things at once. It is comparing how accurately you can get information using both tools. That's just one thing.

That would be biased. Secondly, read again: Google generates 1.14 search result pages per search, suggesting that roughly 86% of time Google presents the data the user is looking for within the first few results. So if you ask Siri "how tall is the empire state building", you'll get a precise answer from Siri (hopefuly) and find that information on the first page of Google results (actually, if you google that you'll now get an answer too). So this is what they're comparing. So again, you're completely missing the point.

And again, just go and read the article.

----------


Another person who missed the whole point of this research...

I'm wondering if we should check your reading comprehension guys...

Wow! Your reading comprehension is so great that you failed to check in with reality. When comparing two things, one should not try to compare apples to oranges. You also failed to understand my comment, so you need to check your comprehension skills.

Some other really stupid comparisons that have been posted lately include" Surface Pro (full windows 8) vs. iPad (mobile OS), Google Map vs Apple Map (UNRELEASED BETA), etc....

 

[Caliber26]

I actually find Siri to be practical on those rare occasions I remember she's there to assist. The things I mostly use it for are placing calls, playing specific songs, setting reminders, and setting alarms. I also like the fact that I can ask it to call a specific business and if it's not in my contacts, it'll do a web search and ask me which of the local search results I want to call. Sure beats having to look it up myself.

 

[JohnDoe98]

Snip

Exactly, thanks for this very clear explanation.

 

[foobarbaz]

Did they really give Google an A+ for correctly recognizing (i.e. copying?) *typed* text?

 

[imgonephishin]

On the accuracy side, Google's 86% rating is derived from comScore data showing that Google generates 1.14 search result pages per search, suggesting that roughly 86% of time Google presents the data the user is looking for within the first few results.

Wait, huh?

This is wrong in so many ways that it makes my head hurt.

1. They used a MEAN of the number of pages used. So, if one person clicks through 101 pages, and 99 people find their result on the first page, it would look like the average person had to click view 2 results pages to find their result.
2. There's no reason that only viewing the first page results is proof that someone found what they're looking for. Perhaps the results were so grossly inaccurate that they simply performed another search or closed the browser window. This happens to me frequently but it's almost always user error in not searching on the right terms. Still, it would appear in this data as me finding my result on the first page.
3. How in the world do you decide that to determine an accuracy percentage all you have to do is subtract the average number of result pages viewed from 2? Huh? See points 1 and 2 as to why this number is not accurate.

Don’t get me wrong. Google’s results are incredibly accurate. In fact, I’d guess that 86% success rate on the first page is grossly underestimating. The point is that the statistical analysis performed is just silly number mangling.

 

[zeromeus]

Thank you!

I don't think he is missing the point of the experiment, I think that he, like me, thinks the point is stupid.

Think about it for more than a millisecond and you'll realize that it's comparing apples and oranges. Google, like every other text based search engine on the web, is simply matching text keywords with a large indexed database built by years of crawling through the web. The search is basically brute force and simple.

Siri is receiving a digital waveform from the microphone, eliminating frequencies from that waveform that are not commonly found in the human voice, translating those remaining frequencies into readable text via a remote server and then interpreting that into a command which it then uses to either search the web using what it assumes to be the correct web search service or perform a function using a local app. That service list includes google search.

The fact of the matter is that the two services are not comparable whatsoever and that for all intents and purposes siri expands upon the abilities of google regardless of the situation.

At least someone here has a brain, some logic, and the will power to put both into good use...

 

[slffl]

Are you joking??? Comparing Google text search to Siri? OMG people will write anything negative about Apple for link bait.

How about comparing these two while driving in your car. Oh ya google has a 90% more chance of crashing.

Or sleeping in your bed. Google has a 100% chance of opening your eyes.

This is getting out of control.

P.S. and usually macrumors does a good job of filtering this crap out, but lately they seem to be enjoying the extra clicks they get from these stories.

 

[Dun Properly]

Man Siri is useless for the most part. Get over it my dude. Apple didn't create man. It's okay it flopped. No one is perfect

Siri may be useless for web searches, but I find it wholly useful for setting up reminders, alarms, task, making lists, texting while driving, etc...

Saying otherwise would be silly.

 

[AppleFan1984]

kids these days are too ghetto for siri. They don't realise you may have to, in life, use the "t" at the end of sentences etc.T

ftfy

 

[Geckotek]

OK, I'm a bit sleep deprived and therefore slow today.....I thought they were comparing Siri to Google's voice recognition solution. Are you guys saying that the Google comparison was done via keyboard and not voice? WTH?? That's not Apples and Organges, that's Apple's and Horses!!! What the heck were they thinking they would accomplish here? Just telling everyone to type it in instead???

 

[Rocketman]

Piper Jaffray's testing was conducted using the iPhone's built-in microphone in the quiet setting and Skullcandy microphone headphones for the outdoor setting, with Siri's ability to correctly comprehend queries falling from 89% to 83% when moving outdoors to the noisy urban setting. On the accuracy side, Google's 86% rating is derived from comScore data showing that Google generates 1.14 search result pages per search, suggesting that roughly 86% of time Google presents the data the user is looking for within the first few results.

So, a 1 year old Beta system is within 2-3 percentage points accuracy as compared to a 10 year old, vetted, and vastly more widely used system, and is hands free, voice activated, and can be used in a noisy urban environment?

Siri sucks!

At failing.

Rocketman

 

[VulchR]

I am sure those of us who actually own an iPhone and actually use Siri will be happier once it is a finished product. Typing in queries into Google is bound to be more accurate both on the input and the search sides of things. However, the degree to which this is true is no doubt dependent on the accuracy of the typist and the nature of the search. Mind you - 'Search Google for [search term]' works fine in Siri.

 

[Belgrano]

It's only being given the slack it is being given as it's by Apple.

Actually, I think more people would view Siri as a questionably-functional gimmick if it were, e.g., a Windows Phone feature.

Instead, it gets the support of a zealous apple fanbase that doesn't actually use it too much.

Edit: Oops! think I misread your post, and we're actually in agreement.

 

[shk718]

I find siri quite accurate in the translation of my voice. where i have problems is when ATT has a week data connection (which in nyc is almost all the time) . then she just spins around. at home on my wifi - shes incredibly fast with the results and 95% accurate. this year i'm going to switch to verizon which is better in NYC - plus w/ 4g (assuming iphone 5 has it) i bet siri will be as amazing outside my apartment.

as far as her abilities... i asked her for a cheep italian restaurant - she came back with a list of shops and then added a reply - to the effect that she cant search by price range. which means she understood exactly what i was looking for but doesn't yet have the data. all apple needs to do is feed her more and more info and she will be amazing. the original siri was amazing - you could have conversations with her - "find a restaurant" - "make it chinese" "book a table there" - and it would follow the conversation exactly - those capabilities are still there - but apple needs to (and they are) add the connections - yelp - open table fandango ... obviously they are having problems getting the deals. i agree i wish apple would do more and more incremental upgrades - not just once a year.

----------

Actually, I think more people would view Siri as a questionably-functional gimmick if it were, e.g., a Windows Phone feature.

Instead, it gets the support of a zealous apple fanbase that doesn't actually use it too much.

i disagree - i use SIRI all the time. calling people, selecting songs, meetings, alarms text messages. once they start adding more capabilities i think SIRI will be unbelievable. my complaint is ATT and the crappy data connections they give me

 

[Blimpy]

Stumbled across this and it makes me worry about Siri. Google's offering is faster and has a much more natural voice. What is so much better about Siri that Apple had to buy them and spend so long working on it? Can it be purely explained by the need for more data centres to support the massive number of queries, or is Apple falling behind?

 

[applephysci]

The most important point of the test has been ingeniously omitted by macrumors.

"Apple's version of Siri is tainted because it's marketing driven, giving preference to commercial sites like Yelp or companies over actual, useful results."

 

[SBlue1]

curious where this is gonna take us in 10 years.

 

[rmwebs]

This is meaningless without testing multiple accents in all languages and having questions posed in different ways!

Pointless to the extreme is what comes to mind.

Siri is more of an experiment than a real feature, it's the interpretation of the sentance that is the exciting part of siri and it's ability to follow a thread.

Google is just a dumb engine with no possible applications other than being a directory of the internet. They have never developed beyond coder style searches for specifics and their engineer backgrounds suggest they never will.

And the awards for "Person who pulls the most bull out of the air" goes to you.

Google have a hell of a lot more information at their disposal than Apple. Information such as world wide weather (yes, Google have their own weather systems), traffic data, business locations, etc - Google have been building up a massive library of info for years. Apple have not. This is why Google's alternative will always beat Siri when it comes to information. Be it questions or local info.

To call Google a "dumb engine with no possible applications other than being a directory of the internet" clearly shows just how little you know about anything.

 

[AustinIllini]

Stumbled across this and it makes me worry about Siri. Google's offering is faster and has a much more natural voice. What is so much better about Siri that Apple had to buy them and spend so long working on it? Can it be purely explained by the need for more data centres to support the massive number of queries, or is Apple falling behind?

YouTube: video

Does it really make you worry though? Siri was equally good at everything except finding pictures of dogs. It certainly wasn't a blowout either way. I thought the display for Kobe Bryant's height was better on Siri, and the reminder was screwed up by the host.

 

[rmwebs]

curious where this is gonna take us in 10 years.

If Google Glass (or Glasses or whatever they call it) is any indication, I'd like to think we will have iHud - a 'heads up display' on a set of glasses, which will replace your touch-screen tablets and phones with 100% voice and visual based control.

For the first time ever these odd sci-fi technologies are actually very close to becoming a reality. Google are looking at launching Glasses in a year or so. I'd hope that Apple at very least have a research team looking into their own options for this...maybe iOS would become eyeOS