What kind of information? The queries are already biased to Google, compromising the results. Also, if it wasn't about voice recognition why are they testing this in quiet and urban environments, reporting the accuracy/comprehension results along the way?
Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Siri Shows Increasing Promise, But Accuracy Falls Short of Google Text Searches
- Thread starter MacRumors
- Start date
- Sort by reaction score
What kind of information? The queries are already biased to Google, compromising the results. Also, if it wasn't about voice recognition why are they testing this in quiet and urban environments, reporting the accuracy/comprehension results along the way?
I doubt it. It's supposed to be a misinterpretation of a real phrase.
I'm wondering the same thing, what phrase was misinterpreted at "pancakes urethra"?
I think the tests would have yielded a higher result if the instructions weren't spoken in a Minnesota accent. Dontcha know?
It's a bit too early to say if flopped. Until it's pulled and no longer part of iOS, or absolutely nobody uses it, it hasn't flopped.
I did read the article, the original one that is. How about you respond to my arguments rather than continue with these red herrings?
People have unreasonable expectations. You're talking to a computer that in itself is quite something. But NO thats not enough, you automatically expect it to respond like a human at first go. Not only that it must recognize any and all accents too.
Even if some accents sound like the individual has some kind of butt plug attached.
I think Siri is already quite amazing at what it does and i know it'll only get better and better. Comparing it to typing in a Google search just seems weird to me. Where is the comparison in that? you're comparing a virtual personal assistant to typing something into Google...and you expect parity?
Its only useless to you because Google didn't make it.
Even if some accents sound like the individual has some kind of butt plug attached.
I think Siri is already quite amazing at what it does and i know it'll only get better and better. Comparing it to typing in a Google search just seems weird to me. Where is the comparison in that? you're comparing a virtual personal assistant to typing something into Google...and you expect parity?
Its only useless to you because Google didn't make it.
People have been talking to computers for well over a decade. It's nothing new. What Siri is supposed to be doing (differently) and well is what is up for debate.
I think Siri is ok. I have little need for it. Most of the time when I use any sort of voice control - it's for dictation of a short message and that doesn't require Siri.
the technology is extremely complex, after having ears since birth humans still quite often miss word and make mistakes in understanding what that person meant to say, SIRI is very new and in BETA.
Even i started SIRI using quite often, its like getting used to.
I found Siri performs quite well for me. Not perfect by any means, but I use it regularly and rarely have issues. A lot of my use is in the car with the A/C on so that's a fairly noisy environment. I use it with the built-in mic, not a BT or wired one.
The main difference I see is that the article seemed to focus on searching. While I occasionally use Siri to search, I mostly use it for things like:
- Read an incoming text message to me while I'm driving and reply to it.
- Take a note (in the car or anywhere else). I even add-on to the note using voice.
- Check my calendar.
- Set up appointments in my calendar.
- Initiate calls
- Set up reminders (I'm especially fond of location-based reminders)
As others posted, accent and diction probably make a significant difference so YMMV.
The main difference I see is that the article seemed to focus on searching. While I occasionally use Siri to search, I mostly use it for things like:
- Read an incoming text message to me while I'm driving and reply to it.
- Take a note (in the car or anywhere else). I even add-on to the note using voice.
- Check my calendar.
- Set up appointments in my calendar.
- Initiate calls
- Set up reminders (I'm especially fond of location-based reminders)
As others posted, accent and diction probably make a significant difference so YMMV.
I don't use it as extensively as others, but I think if you tried, you could easily find uses for it.
My favorite uses that work well are:
1) Turning on the countdown timer when I'm grilling meat
2) Setting reminders
3) Setting appointments
4) The occasional web search (I usually type these in though...out of habit)
Saying "Set timer to 4 minutes" is never misinterpretted (unless I stutter). Setting reminders are occasionally misinterpretted, but I always know what I meant to they get the job done. My favorites are things like "Remind me to return my Redbox movie when I leave work" or something along those lines.
On the frustrating side, setting appointments often confuses me as I don't know the right words to use. I can say "Set an appointment for Anthony's Wedding on September 2nd at noon" and the appointment just comes up with a subject of "Appointment". Any suggestions on getting this right would be helpful. Apple really needs to work on Siri's logic on that one.
I'm really looking for iOS 6 and hoping for some real improvments as well as the ability to interface with 3rd party apps.
This is the most stupid test a person can come up with....
It's like giving my students some oral instructions Tuesday and write the directions on the board on Wednesday and saying that students have better comprehension skills on Wednesdays than they do on Tuesday.
Oral queries will always be less accurate than written queries, just as this analyst had done. Of course oral queries from Siri would be less accurate than text input queries done on Google.
This ONLY PROVES that Google FAILED.
It's like giving my students some oral instructions Tuesday and write the directions on the board on Wednesday and saying that students have better comprehension skills on Wednesdays than they do on Tuesday.
Oral queries will always be less accurate than written queries, just as this analyst had done. Of course oral queries from Siri would be less accurate than text input queries done on Google.
This ONLY PROVES that Google FAILED.
Fine, if you want to embarrass yourself...
On the accuracy side, Google's 86% rating is derived from comScore data showing that Google generates 1.14 search result pages per search, suggesting that roughly 86% of time Google presents the data the user is looking for within the first few results.
So this the point. Will you get information quicker using Siri or googling? So you've completely missed the point here.
It's not comparing two things at once. It is comparing how accurately you can get information using both tools. That's just one thing.
That would be biased. Secondly, read again: Google generates 1.14 search result pages per search, suggesting that roughly 86% of time Google presents the data the user is looking for within the first few results. So if you ask Siri "how tall is the empire state building", you'll get a precise answer from Siri (hopefuly) and find that information on the first page of Google results (actually, if you google that you'll now get an answer too). So this is what they're comparing. So again, you're completely missing the point.
And again, just go and read the article.
----------
Another person who missed the whole point of this research...It's like giving my students some oral instructions Tuesday and write the directions on the board on Wednesday and saying that students have better comprehension skills on Wednesdays than they do on Tuesday.
Oral queries will always be less accurate than written queries, just as this analyst had done. Of course oral queries from Siri would be less accurate than text input queries done on Google.
This ONLY PROVES that Google FAILED.
I'm wondering if we should check your reading comprehension guys...
So they're saying that a newer technology performs better than an older +6 month old one? STOP THE PRESSES!
Nope. This has nothing to do with Siri vs Google Now.
Guys, read the whole article goddamit!
Well, of course. The article relates to accuracy of speech. Siri has potential to perform other tasks. However, if speech accuracy falls shorts, it doesn't matter much. One hundred percent of everything I say, including using Google navigation and maps, is understood and interpreted correctly to do what I wanted. Siri on my wifes iphone 4s has potential to be cooler, but it often mistakes what I say, and can be painfully slow to respond even at home on 30 Mbps wifi. Still maybe iOS 6 will bring better accuracy and we could all certainly benefit from that. **crossing fingers**
I just googled for "what spices are in Lasagna". Google says there are 2,790,000 search results. None of them will just answer the question. Well, none of the first five, I didn't check the other 2,789,995.
Really? An experiment? Funny, I don't see "experiment" or even "beta" being used on their commercials.
It's one thing to say that Siri isn't great yet. It isn't. It gets it right for me about 65% of the time when I'm doing general queries.
However, I've actually used Android devices and Voice Actions on Google isn't great either. Neither is Google Maps for that matter. It's just that Android users won't openly complain. I get about the same level of accuracy(maybe a tiny bit more accurate?).
The oddity is that Google is never held accountable for things such as routes I would never take or subway directions that tell me to take five trains when I can just take two.
Apple should be held accountable because they heavily promoted a feature in beta but this idea that Google's version is dramatically better is simply ridiculous, especially when Apple is using Nuance for voice recognition on the back end.
For once, I haven't read the original article, but it sounds like the tests need to be broken up into three areas:
1) How well the microphones work on various phones. That is, test the same recorded speech in various environments.
2) How well the speech recognition works (i.e. Nuance versus Google's unknown engine). That is, speech -> text for the search.
3) How well the resulting search works.
PS. Okay, that's engineer thinking. A user simply cares how it works end to end
1) How well the microphones work on various phones. That is, test the same recorded speech in various environments.
2) How well the speech recognition works (i.e. Nuance versus Google's unknown engine). That is, speech -> text for the search.
3) How well the resulting search works.
PS. Okay, that's engineer thinking. A user simply cares how it works end to end
Last edited:
I agree. What's more, when something is in beta or open beta, it's typically something optional that you have to install or otherwise elect to participate in order to have it. Example: The early beta of FaceTime on Mac. I don't think Siri is an "optional" feature in that it's installed whether you want it or not.
