Register FAQ / Rules Forum Spy Search Today's Posts Mark Forums Read
Go Back   MacRumors Forums > News and Article Discussion > MacRumors.com News Discussion

Reply
 
Thread Tools Search this Thread Display Modes
Old Dec 10, 2013, 10:10 AM   #1
MacRumors
macrumors bot
 
Join Date: Apr 2001
Siri's Accuracy Continues to Improve as Apple Works to Reduce Reliance on Google




Piper Jaffray analyst Gene Munster, who has regularly assessed Siri's accuracy in terms of correctly interpreting and answering queries, has issued the latest version of his Siri report card, noting that Siri has continued to improve under iOS 7, particularly in terms of being able to properly interpret questions being asked.
Quote:
The primary improvement between Siri in iOS 6 and iOS 7 came from its performance in noisier areas, where Siri was able to translate better through a noisy environment (94% vs. 83% correct in Aug iOS 6). Based on our testing, iOS 7 Siri's net ability to answer questions (factors in questions not correctly understood) improved to 79% from 77% in August with iOS 6.
The improvements in Siri's understanding come alongside but separate from Apple's continued efforts to reduce its reliance on Google, as evidenced by Siri's shift to emphasize Bing, Wikipedia, and WolframAlpha results. In Munster's most recent testing, only 4 percent of Siri's answers relied on Google, compared to 17 percent in August testing and 27 percent in December 2012.

Click for larger
Munster also compares Siri to Google Now, finding that the two services are now essentially tied in effectiveness with a "C+" score after Siri held a substantial lead in the year-ago testing. In the latest testing, Google Now held small leads in content areas such as commerce and information, while Siri performed significantly better for operating system controls.


Article Link: Siri's Accuracy Continues to Improve as Apple Works to Reduce Reliance on Google
MacRumors is offline   0 Reply With Quote
Old Dec 10, 2013, 10:12 AM   #2
OzyOly
macrumors 6502
 
Join Date: Jun 2009
Location: UK
It still does like Northern (British) English though...
__________________
| MacBook Air | iPhone 5 | iPad rMini LTE | 1Gbps Internet |
OzyOly is offline   1 Reply With Quote
Old Dec 10, 2013, 10:13 AM   #3
NightFox
macrumors 65816
 
NightFox's Avatar
 
Join Date: May 2005
Location: Shropshire, UK
So even though its accuracy at understanding has gone up, the accuracy of its answers has actually got worse?
__________________
2013 Mac Pro 6x3.5GHz :: MacBook Pro 13" 2.53GHz :: iPad 3 64GB :: iPho... Oh, you've gone.
NightFox is offline   12 Reply With Quote
Old Dec 10, 2013, 10:14 AM   #4
roadbloc
macrumors 604
 
roadbloc's Avatar
 
Join Date: Aug 2009
Location: UK
Regretfully, Siri still doesn't understand me. Good job I have no real need for voice control.
__________________
roadbloc is offline   3 Reply With Quote
Old Dec 10, 2013, 10:15 AM   #5
chirpie
macrumors 6502a
 
Join Date: Jul 2010
I don't know, that net percentage increase (4% better) year over year seems kind of low to me. Am I the only one who feels that way? Not my field of study, so I'm not entirely sure if this is supposed to be impressive.
chirpie is offline   6 Reply With Quote
Old Dec 10, 2013, 10:15 AM   #6
Amazing Iceman
macrumors 68020
 
Amazing Iceman's Avatar
 
Join Date: Nov 2008
Location: Florida, U.S.A.
How could Un-Controlled Environment results for iOS 7 be 94% and 7%, which adds to 101% ???

The same happens to Controlled Environment for iOS 6.
__________________
17" MacBook Pro (2007) iPad Air WiFi+Cell 128 GB iPhone 5s 64 GB T-Mobile AppleTV 2
Follow @AmazingIceman for useful tech info and more (mention MacRumors).
Amazing Iceman is offline   0 Reply With Quote
Old Dec 10, 2013, 10:16 AM   #7
Ventilatedbrain
macrumors regular
 
Join Date: Nov 2012
I may be in the monitory here but I really find Siri to be very helpful especially when biking.. i ask it to read my texts and call someone etc .. Google now on my nexus 4 feels a bit distant
Ventilatedbrain is offline   3 Reply With Quote
Old Dec 10, 2013, 10:17 AM   #8
AppleMacNerd
macrumors regular
 
Join Date: Jan 2013
Location: United Kingdom
Quote:
Originally Posted by Amazing Iceman View Post
How could Un-Controlled Environment results for iOS 7 be 94% and 7%, which adds to 101% ???

The same happens to Controlled Environment for iOS 6.
I also noticed this!
__________________
Mid 2011 13" MBA i5 1.7Ghz 4GB RAM 128GB SSD
16GB iPhone 5S - Space Grey
AppleMacNerd is offline   1 Reply With Quote
Old Dec 10, 2013, 10:17 AM   #9
steve-p
macrumors 68000
 
Join Date: Oct 2008
Location: Newbury, UK
If the purpose really is to reduce reliance on Google, then maybe the effort would be better spent working on a search engine I can type stuff into, not just talk to.
steve-p is offline   2 Reply With Quote
Old Dec 10, 2013, 10:17 AM   #10
chirpie
macrumors 6502a
 
Join Date: Jul 2010
Quote:
Originally Posted by Amazing Iceman View Post
How could Un-Controlled Environment results for iOS 7 be 94% and 7%, which adds to 101% ???
I'll wager there's some overlap leeway. Or the rounding up/down just created a slight anomaly on the percentages.
chirpie is offline   1 Reply With Quote
Old Dec 10, 2013, 10:20 AM   #11
NachoGrande
macrumors 6502
 
Join Date: Mar 2010
I know one phrase is get correct 100% of the time. SIRI, you suck! After I get the message that SIRI cannot help me right now.
__________________
27" iMac(2010) // 13" MBA (2011) // iPad // iPad 2 // iPad 3 // iPhone 4s // ATV2
NachoGrande is offline   0 Reply With Quote
Old Dec 10, 2013, 10:21 AM   #12
Michael CM1
macrumors 601
 
Join Date: Feb 2008
When I got iOS 7 I did notice a pronunciation correction. There's a town nearby called Braselton, pronounced "Brah-zel-ton." Siri had been pronouncing it "Bray-zel-ton." It's correct on TBT directions now. But she still can't pronounce "sugar" like there's an h after the s.
__________________
Mid-2010 21.5" iMac; silver iPhone 5S 32GB; white iPad Air 32 GB; third-gen TV
Michael CM1 is offline   0 Reply With Quote
Old Dec 10, 2013, 10:25 AM   #13
HurtinMinorKey
macrumors regular
 
Join Date: Jan 2012
This, and the eventual analog on Windows, will be the death of Google. Or at the very least, a breakup of its monopoly on search ads.
HurtinMinorKey is offline   1 Reply With Quote
Old Dec 10, 2013, 10:26 AM   #14
Xenomorph
macrumors 6502a
 
Join Date: Aug 2008
Location: St. Louis
Who made those charts?

Randomly switched fonts and terrible kerning? Gross.
Xenomorph is offline   3 Reply With Quote
Old Dec 10, 2013, 10:30 AM   #15
lil' brudder
macrumors 6502
 
Join Date: Jan 2007
Location: Minneapolis, MN
There is no way these charts are accurate. Even when I speak as clearly as possible and enunciate every syllable, Siri understands me about 40% of the time.
__________________
Ivy Bridge 11.6" MBA, 1.7GHz i5, 256GB SSD, 8GB RAM
Black 32 GB iPhone 5 - Verizon
Retired: White MacBook 1.83GHz Core Duo, 2GB RAM, 320GB HD
lil' brudder is offline   10 Reply With Quote
Old Dec 10, 2013, 10:30 AM   #16
pasqualino
macrumors member
 
Join Date: Oct 2013
in other news, company b is trying to improve their products.
pasqualino is offline   0 Reply With Quote
Old Dec 10, 2013, 10:31 AM   #17
HenryDJP
macrumors 68000
 
Join Date: Nov 2012
Location: United States
Quote:
Originally Posted by roadbloc View Post
Regretfully, Siri still doesn't understand me. Good job I have no real need for voice control.
The one thing I will greatly defend Apple and Siri for is trying to understand people's voices. People either have a strong accent, speak with a lisp, speak too fast, speak too slow, speak with a low register voice, have an annoying voice, mumble their words, garble their words and most of all they try and use Siri in a noisy atmosphere along with any of the above.

No matter what, it's still an electronic device and people just have to learn to speak properly based on the source or just don't use as you've appeared to do. You say you have no need for voice control, it's not just about voice control. Many times I need to look up sports scores, it's much faster and easier just to ask Siri the game stats than typing them in or opening an app and searching.
HenryDJP is offline   3 Reply With Quote
Old Dec 10, 2013, 10:32 AM   #18
Popeye206
macrumors 68020
 
Popeye206's Avatar
 
Join Date: Sep 2007
Location: NE PA USA
Quote:
Originally Posted by OzyOly View Post
It still does like Northern (British) English though...
I use to work with someone from Glasgow… heck, I couldn't understand them half the time…. especially after a few pints!
Popeye206 is offline   1 Reply With Quote
Old Dec 10, 2013, 10:34 AM   #19
HenryDJP
macrumors 68000
 
Join Date: Nov 2012
Location: United States
Quote:
Originally Posted by lil' brudder View Post
There is no way these charts are accurate. Even when I speak as clearly as possible and enunciate every syllable, Siri understands me about 40% of the time.
I get 90% accuracy, I don't know what's going on with you.
HenryDJP is offline   13 Reply With Quote
Old Dec 10, 2013, 10:34 AM   #20
skoosht
macrumors newbie
 
Join Date: Jul 2011
Location: Los Angeles, CA, USA
Google search works nearly 100% of the time for me on my iPhone 4. Siri 0% hahaha....oh wait, apple says my phone can't handle voice search. tsk tsk apple.
skoosht is offline   5 Reply With Quote
Old Dec 10, 2013, 10:36 AM   #21
zombierunner
macrumors 6502
 
Join Date: Jan 2011
Google voice does better than Siri if you have an accent .. I tried both .. Google seems to work more times than Siri .. Hopefully Apple can sort this out
zombierunner is offline   4 Reply With Quote
Old Dec 10, 2013, 10:37 AM   #22
NMBob
macrumors regular
 
Join Date: Sep 2007
Is this a problem?

http://www.youtube.com/watch?v=Q3bdXctq7DM

NMBob is offline   0 Reply With Quote
Old Dec 10, 2013, 10:37 AM   #23
Simmias
macrumors member
 
Join Date: May 2010
I have had the opposite experience. After seeing steady improvement in Siri, it has recently gotten worse and become almost unusable for me.

Specifically, I use Siri a lot to play music in the car. I will say "Play Song X" or "Play Album Y". Siri used to be pretty good at hearing the name of the song and finding it in my music library. Now it frequently hear the names incorrectly. Even when it does hear the correct name, it will play a song with a completely different name or simply say "Sorry, I couldn't find that in your music". What do you mean you couldn't find David Bowie in my music??? It's right there!

This regression is quite infuriating. Also, Siri still often fails to respond ("Sorry, there's something wrong") or take 20+ seconds to respond with the wrong answer. I believe we need to give all of Apple's web services time to gather data and make iterative improvements, but I have very little tolerance when they actually get worse over time.
Simmias is offline   7 Reply With Quote
Old Dec 10, 2013, 10:38 AM   #24
NightFox
macrumors 65816
 
NightFox's Avatar
 
Join Date: May 2005
Location: Shropshire, UK
Quote:
Originally Posted by HenryDJP View Post
The one thing I will greatly defend Apple and Siri for is trying to understand people's voices. People either have a strong accent, speak with a lisp, speak too fast, speak too slow, speak with a low register voice, have an annoying voice, mumble their words, garble their words and most of all they try and use Siri in a noisy atmosphere along with any of the above.

No matter what, it's still an electronic device and people just have to learn to speak properly based on the source or just don't use as you've appeared to do. You say you have no need for voice control, it's not just about voice control. Many times I need to look up sports scores, it's much faster and easier just to ask Siri the game stats than typing them in or opening an app and searching.
Never defend a technology! Technology is meant to develop to serve the requirements/limitations of existing human behaviour; human behaviour shouldn't be expected to adapt to meet the requirements/limitations of technology.
__________________
2013 Mac Pro 6x3.5GHz :: MacBook Pro 13" 2.53GHz :: iPad 3 64GB :: iPho... Oh, you've gone.
NightFox is offline   8 Reply With Quote
Old Dec 10, 2013, 10:39 AM   #25
JHankwitz
macrumors 65816
 
Join Date: Oct 2005
Location: Wisconsin
Send a message via AIM to JHankwitz
Quote:
Originally Posted by Amazing Iceman View Post
How could Un-Controlled Environment results for iOS 7 be 94% and 7%, which adds to 101% ???

The same happens to Controlled Environment for iOS 6.
Haven't you ever heard of the "new math"? It was all the rage back in the 70's and 80's. The kids from that era now work for the media, spewing forth whatever numbers they get without analysis or question.
JHankwitz is offline   0 Reply With Quote

Reply
MacRumors Forums > News and Article Discussion > MacRumors.com News Discussion

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 02:43 AM.

Mac Rumors | Mac | iPhone | iPhone Game Reviews | iPhone Apps

Mobile Version | Fixed | Fluid | Fluid HD
Copyright 2002-2013, MacRumors.com, LLC