Siri Shows Increasing Promise, But Accuracy Falls Short of Google Text Searches

Discussion in ' News Discussion' started by MacRumors, Jun 29, 2012.

  1. macrumors bot


    Apr 12, 2001

    In what should not be a surprise given Siri's beta status and its use of new technology for voice interaction with mobile devices, a study from Piper Jaffray analyst Gene Munster finds that Siri's abilities are still eclipsed by the traditional text-entry Google searches that have been in place for years.

    The study included presenting Siri with 800 queries in each of two environments, a quiet room and a busy urban street, and then assessing Siri's ability to correctly interpret the queries and provide correct responses. Specifically addressing the outdoor testing, Munster notes:
    But while Siri has some room for improvement, Munster acknowledges that the technology is only in its infancy and he remains "optimistic about its future" with an eye toward iOS 6 as bringing substantial improvements.


    Piper Jaffray's testing was conducted using the iPhone's built-in microphone in the quiet setting and Skullcandy microphone headphones for the outdoor setting, with Siri's ability to correctly comprehend queries falling from 89% to 83% when moving outdoors to the noisy urban setting. On the accuracy side, Google's 86% rating is derived from comScore data showing that Google generates 1.14 search result pages per search, suggesting that roughly 86% of time Google presents the data the user is looking for within the first few results.

    Piper Jaffray also analyzed the sources for Siri's results, finding that Google provided 60% of the answers, with Yelp and other sources filling in the remainder.
    Munster estimates that Google's share of Siri results will drop to 48% with the launch of iOS 6 as navigation, sports, and movie queries shift over to Apple and its new partners.

    Article Link: Siri Shows Increasing Promise, But Accuracy Falls Short of Google Text Searches
  2. macrumors member


    Nov 30, 2011
    Santa Barbara, California
  3. macrumors regular


    Sep 7, 2010
    Ozone Park, New York
    Many people have bashed siri for not being accurate/ helpful. Honestly when siri came out I never saw myself using it a whole lot. But now, I use it regularly on a daily basis. It has come a long way even though it was just released a couple months ago.
  4. macrumors demi-god


    Oct 20, 2011
    Austin, TX
    I feel like Apple should do more delta updates to Siri. Pulling it off beta should be the last major publicized update to Siri. After that, just update it periodically.
  5. macrumors G4

    Small White Car

    Aug 29, 2006
    Washington DC
    I wonder if Apple's just going to continue with small, invisible background updates to Siri.

    I kind of wonder if they're going to hold back on some of the big updates and have a big "1.0 announcement" when iOS 6 comes out. I sort of think that method might be a good idea since people's opinion of Siri seem to be sagging.
  6. macrumors member

    Nov 15, 2008
    Siri is not on the same level as Google voice. Just saying.
  7. macrumors 65816

    Oct 4, 2005
    Toronto, Canada
    Not totally surprising. Google's been working on their search algorithms for years.

    But the only fair comparison is when you ignore the times when Siri misinterprets what you're saying. Siri has vocal input, but after it processes your speech into text, it is still doing a text based search.

    With iOS 6, it seems Siri is improving. And obviously Apple will continue to improve it's ability to answer correctly and more accurately.

    In the end though, Siri is only as good as its sources.
  8. macrumors 6502a

    Jun 25, 2008
    I'm using Siri to post this message and, as you can see, it's 100% a cure writ and pancakes urethra.
  9. macrumors 6502a

    Jun 3, 2012
    This is meaningless without testing multiple accents in all languages and having questions posed in different ways!

    Pointless to the extreme is what comes to mind.

    Siri is more of an experiment than a real feature, it's the interpretation of the sentance that is the exciting part of siri and it's ability to follow a thread.

    Google is just a dumb engine with no possible applications other than being a directory of the internet. They have never developed beyond coder style searches for specifics and their engineer backgrounds suggest they never will.
  10. macrumors newbie

    Jun 17, 2008
    Siri is substandard

    To me, there's no two ways about it. Uncharacteristically, Apple releases a "beta" product, repeatedly invoking that fact as if it were an excuse.

    Which would be fine, if they charged a "beta-worthy" price for iPhone 4S.

    Sorry, but it's been well over half a year, and Siri STILL underperforms. Badly.

    In my experience and in all kinds of situations and environments, Siri works less than 50% of the time. More like a third of the time. I've pretty much given up on it.

    We all love Apple, but to me it's hard not to state the obvious. Siri is (so far) a flop, in terms of living up to its profusely advertised promise. It is not even close to the usual high standards of Apple/SJ. And if he were alive now, I think he would be blowing a gasket about it.
  11. macrumors 68030


    Oct 16, 2007
    What Google showed yesterday with offline support was very impressive, I agree.
  12. macrumors newbie

    Sep 2, 2010
    Accurate ? Speak yes, not Siri !

    I tried since a week and I can say you that, in French as in English, when I ask from the keyboard, it works really really well, even with background music or tv. But when I ask the same phrase to Siri... it's very often wrong...

    I think then Apple can after the beta (and dev prev) just update it to take the "speech" more correctly...
  13. macrumors member

    Jun 11, 2010
  14. macrumors 6502a

    Sep 3, 2009
    Siri is a great feature, but about 50% of the time when I try to use it, I end up canceling while she's thinking and just doing what I needed to do manually, since it's faster. I don't want to wait for 30 seconds to find out if it's going to wake me up at the right time, or at all in the morning. Luckily, it's software and can be improved.
  15. macrumors 68040


    Sep 24, 2010
    Kids these days are too Ghetto for Siri. They don't realise you may have to, in life, use the "T" at the end of sentences etc.

    I'm by no means posh but Siri understands me quite well. Obviously not so much if you're not a slack jawed idiot.
  16. macrumors 68020

    May 1, 2009
    What terrible reporting. Why wasn't it mentioned when giving the comparative data that the Google queries were typed in? Of course typed text will be more accurate than voice recognition. Highly misleading dissemination of data here.
  17. macrumors member

    Nov 15, 2008
    lol you're such a hater man. I hope Apple is paying you.
  18. macrumors 6502a

    May 18, 2012
    far away from an Apple store
    I think you've been living under a rock if you seriously believe in this...
  19. macrumors member

    Nov 15, 2008
    Thank you. See, we can all get along.
  20. macrumors 6502a

    May 18, 2012
    far away from an Apple store
    Siri fails even more when it comes to other accents, so you don't want to go there.
  21. macrumors G3


    Jul 22, 2008
    Doesn't using the headphones disable the built in noise cancelling? Seems like results might have been better if they used the built in mic.
  22. macrumors 6502a


    Jun 15, 2010
    Nominated for "Post of the Month" award.
  23. macrumors 65816

    Oct 4, 2005
    Toronto, Canada
    Hopefully Apple adds offline processing for local actions. Could be an additional announcement with iOS 6 in the fall? Maybe exclusive to the new iPhone... that'd be lame though. When Siri exits beta, it should definitely have local processing for the local commands.
  24. macrumors regular

    Jun 29, 2008
    THe article's premise makes no sense

    Munster's assessment compares two completely different resources, a search engine and a personal assistant.

    It compares them only on search (since that is all that Google does).

    Siri does search, but of a limited sort (e.g., "what is that plane overhead?" "NCA178 Boeing 747-400 at 18,000 feet."). Google does not do that. Nor does it set alarms or remind you to get gas when you leave work.

    If you want Siri to do a web search, you can just ask for it.
    "Google what team does Peyton Manning play for?"
    "Search the Web for what spices are in Lasagna?"
    "Yahoo when is the next Haley's comment?"*
    * The comment: 'Haven't we already seen that taily thing?"

    There is no learning curve.
  25. macrumors member

    Nov 15, 2008

Share This Page