Apple's Machine Learning Has Cut Siri's Error Rate by a Factor of Two

Discussion in 'MacRumors.com News Discussion' started by MacRumors, Aug 24, 2016.

  1. MacRumors macrumors bot

    MacRumors

    Joined:
    Apr 12, 2001
    #1
    [​IMG]


    Steven Levy has published an in-depth article about Apple's artificial intelligence and machine learning efforts, after meeting with senior executives Craig Federighi, Eddy Cue, Phil Schiller, and two Siri scientists at the company's headquarters.

    [​IMG]

    Apple provided Levy with a closer look at how machine learning is deeply integrated into Apple software and services, led by Siri, which the article reveals has been powered by a neural-net based system since 2014. Apple said the backend change greatly improved the personal assistant's accuracy.
    Alex Acero, who leads the Siri speech team at Apple, said Siri's error rate has been lowered by more than a factor of two in many cases.
    Acero told Levy he was able to work directly with Apple's silicon design team and the engineers who write the firmware for iOS devices to maximize performance of the neural network, and Federighi added that Apple building both hardware and software gives it an "incredible advantage" in the space.
    Apple's machine learning efforts extend far beyond Siri, as evidenced by several examples shared by Levy:
    Another product born out of machine learning is the Apple Pencil, which can detect the difference between a swipe, a touch, and a pencil input:
    On the iPhone, machine learning is enabled by a localized dynamic cache or "knowledge base" that Apple says is around 200MB in size, depending on how much personal information is stored.
    Apple, for example, uses its neural network to capture the words iPhone users type using the standard QuickType keyboard.
    Apple insists that much of the machine learning occurs entirely local to the device, without personal information being sent back to its servers.
    The full-length article on Backchannel provides several more details about how machine learning and artificial intelligence work at Apple.

    Article Link: Apple's Machine Learning Has Cut Siri's Error Rate by a Factor of Two
     
  2. Santabean2000 macrumors 68000

    Santabean2000

    Joined:
    Nov 20, 2007
    #2
    Siri stills sucks for me... still doesn't work well enough to be useful.
     
  3. GfPQqmcRKUvP macrumors 68040

    GfPQqmcRKUvP

    Joined:
    Sep 29, 2005
    Location:
    Terminus
    #3
    Not in my experience. Siri is absolute trash compared to Google Now and Cortana. Apple should be embarrassed by how stagnant it has been and the repeated, correctable errors it makes. No amount of public relations whitewashing can fix it.
     
  4. SandboxGeneral Moderator emeritus

    SandboxGeneral

    Joined:
    Sep 8, 2010
    Location:
    Detroit
    #4
    This is a feature that I have found to work quite well for me and I like it a lot. Its so uncanny that it can happen without me having to search for an app at a location many times.
     
  5. freepomme Suspended

    Joined:
    Oct 30, 2015
    Location:
    Boston, MA
  6. Relentless Power macrumors Core

    Relentless Power

    Joined:
    Jul 12, 2016
    #6
    It's only my opinion, but Siri struggles with the most basic tasks. I do appreciate Siri and I am hoping for advancements to keep progressing, making her more efficient. But over all, Siri has improved over the years.
     
  7. LordQ Suspended

    LordQ

    Joined:
    Sep 22, 2012
    #7
    I would love more functionality and less jokes. Yesterday I asked her to show me Foxes music on iTunes and had the common "I don't understand what you mean by blabla...".

    If they added Translations like Cortana I'd be a very happy chap, I watch a lot of video recipes and would love to ask Siri stuff like "How do you say shallots in Spanish".
     
  8. smacrumon macrumors 68030

    smacrumon

    Joined:
    Jan 15, 2016
    #8
    I use Siri a lot for dictation and Siri has been improving over the years. A great feature when introduced with iPhone 4S, and has only become better since.
     
  9. 840quadra Moderator

    840quadra

    Staff Member

    Joined:
    Feb 1, 2005
    Location:
    Twin Cities Minnesota
    #9
    I have a different experience. Siri is more understanding of difficult to pronounce names, doesn't require corrections as often, and has much less errors when I am using BlueTooth in my car. I rarely have to talk in a robot voice for it to understand me, where as when it first came out, it was a standard practice.
     
  10. NY Guitarist macrumors 65816

    NY Guitarist

    Joined:
    Mar 21, 2011
    #10
    I like Siri on the iPhone when it works, but at times it totally gets things very wrong that it previously got right.

    What really gets me ticked off is how Siri on ATV is worse than useless almost every time I try to make use of it.
     
  11. keysofanxiety macrumors 604

    keysofanxiety

    Joined:
    Nov 23, 2011
    #11
    And yet it still has no offline functionality, even if the learning is localised; well from what I can see, anyway. An iPhone 3GS managed offline voice functions like "call Tom". Why the heck can't Siri? Phone, text, open an app -- these are not commands that need to be bounced off a server.

    It's fine if you're at Apple HQ, testing Siri on a 10Gb/s Internet connection. But when you're on the road with flaky 3G signal, it's the last thing you need.
     
  12. Oblivious.Robot, Aug 24, 2016
    Last edited: Aug 24, 2016

    Oblivious.Robot macrumors 6502a

    Oblivious.Robot

    Joined:
    Sep 15, 2014
    #12
    Yeah, no.
    Siri is still pretty much useless for me, although better than before, but still meh with her current capabilities.

    image.jpeg

    Edit - For the screen brightness part, after some retries I found out that it does work, but you have to say it in one sentence as Siri forgets about it in the next. :confused:
     
  13. ike1707 macrumors 6502

    Joined:
    Jan 20, 2009
    #13
    I take bigger issue with things like the "play the rest of this album" command instead shuffling my entire catalog of an artist, and saying "audiobook" to play my audiobook one day, and the next day requiring "play audiobook" to even understand my command.

    Speech recognition is great, important really, but I would appreciate them fixing all the other sh*t she doesn't do right too.
     
  14. npmacuser5 macrumors 6502a

    npmacuser5

    Joined:
    Apr 10, 2015
    #14
    Others, like Amazon, figured out that 6 microphones and noise canceling are essential for good voice recognition. Like talking to a person who has limited hearing, "what did you say?". The old garbage in garbage out. A suggestion for Apple, have Siri bring up a text screen of what she heard after so many tries. Amazon has a feature that allows the user to see what Alexa heard, make a correction, and feedback to Amazon. Nice to see more research and creative thinking going on. Voice recognition has a good future.
     
  15. jsclem02 macrumors newbie

    jsclem02

    Joined:
    Aug 24, 2016
    Location:
    Louisville, KY
    #15
    The struggle with Siri is still very real.
     

    Attached Files:

  16. wizard macrumors 68040

    Joined:
    May 29, 2003
    #16
    Well the hype machine is in full gear. Honestly I haven't used Siri in months but it was no able worst than it was for me a few months ago. Simple things like driving along asking for a map to destination xx just fails miserably. It is like Siri doesn't comprehend that asking for a map to xx means popping up the map app with the directions ready to go. Siri has simply been terrible for me.

    No then interesting thing here is that they are claiming machine learning techniques on the device. This makes me wonder how far away we are to an A series chip from Apple wth built in neural network technology. This would do more to advance the platform than another core as it would dramatically improve machine learning. So the question is Apple, how soon will the A series chips have built in neural network hardware?
     
  17. themachugger macrumors member

    Joined:
    Aug 26, 2010
    #17
    If Siri is learning, she's learning disabled. She constantly thinks I said "Skip 32nd" and goes looking for a movie called "Skip 32nd" on my Apple TV when it should be OBVIOUS I want it to SKIP THIRTY SECONDS. I could forgive her making the mistake once or twice but she never seems to figure it out.
     
  18. EdT, Aug 24, 2016
    Last edited: Aug 24, 2016

    EdT macrumors 65816

    EdT

    Joined:
    Mar 11, 2007
    Location:
    Omaha, NE
    #18
    Siri on my phone and autocorrect on any Mac device-IPad, MacBook or desktop- has gone from somewhat useful to irritating. When it gets something wrong it doesn't accept corrections , usually insisting on changing it back to the wrong word even if I go back and type in the correct word or spelling. My niece Caitlin is not Kaitlyn, but Siri insists on spelling it that way and autocorrect will change it back if I manually change it. I have put 'Caitlin' as a shortcut for 'Kaitlyn' but that doesn't fix the problem. And typing this message on an iPhone 6 meant retyping Kaitlyn several times, and the last, mis-spelled time above I didn't go back and correct. There are other words, spellings, and phrases that Siri seldom if ever gets right, and typing corrections do not seem to improve her accuracy.
     
  19. TheShadowKnows! macrumors 6502

    TheShadowKnows!

    Joined:
    Sep 30, 2014
    Location:
    National Capital Region
    #19
    "Apple's Machine Learning Has Cut Siri's Error Rate by a Factor of Two"

    error-rate = power(infinity,.5) = infinity
     
  20. thisisnotmyname, Aug 24, 2016
    Last edited by a moderator: Aug 25, 2016

    thisisnotmyname macrumors 68000

    thisisnotmyname

    Joined:
    Oct 22, 2014
    Location:
    known but velocity indeterminate
    #20
    I agree. On the phone she works well for me. On the AppleTV it seems like she goes to sleep and the first time I try to interact with her such as "what did he just say?" it lags so long that the ten second rewind is well past whatever I wanted to hear. And then there's the lack of Siri search through home sharing content...
     
  21. vp719 macrumors regular

    Joined:
    Jan 13, 2007
    #21
    This happens all the time, drives me nuts.
     
  22. wigby macrumors 65816

    Joined:
    Jun 7, 2007
    #22
    This story is a great example of Apple's scale problem. Since they do so much international business, when they say they've improved Siri by 2X, it's true. The problem is that anecdotally, for most of us, those improvements are not perceived. I only use Siri in English and for a very small set of tasks so I see no improvement and I assume most of us are in the same boat. If you surveyed other Siri users across 100 countries, I bet we would see a bigger pattern of improvement.
     
  23. Rogifan macrumors P6

    Rogifan

    Joined:
    Nov 14, 2011
    #23
    This all seems a little defensive to me. The flavor of the year according to the tech press is AI/ML so now Apple's out there saying we do this too!
     
  24. jbachandouris macrumors 601

    jbachandouris

    Joined:
    Aug 18, 2009
    Location:
    Upstate NY
    #24
    And apparently, 'Siri Sleep' is too hard of a command for the Apple TV.

    2 X 0 = 0
     
  25. samcraig macrumors P6

    Joined:
    Jun 22, 2009
    Location:
    USA
    #25
    Indeed. I also think the terms AI/ML has been diluted. It's becoming the new "cloud" and "big data"
     

Share This Page