Apple Computer, OS X®, iOS® and the Net Promoter® Score (NPS®)

Discussion in 'macOS' started by grahamperrin, Nov 15, 2014.

  1. grahamperrin, Nov 15, 2014
    Last edited: Nov 15, 2014

    grahamperrin macrumors 601

    grahamperrin

    Joined:
    Jun 8, 2007
    #1
    Scores – based on votes in MacRumors Forums


    Is it normal to find so much detraction following the initial release of an operating system?

    [​IMG]

    Related reading – MacRumors

    Are You Going To Install Yosemite?

    Are voters in MacRumors Forums more optimistic – about Apple products – than voters at large?

    10.9 to be the last of the old school ?

    Related reading – elsewhere

    Study: Apple customers are by far the most loyal of any computer company – MacDailyNews - Welcome Home (2006-07-11, highlights)

    The Ultimate Question | Computerworld (2006-07-24)

    Net Promoter Score (NPS) Criticisms and Best Practices (2009-04-28, highlights)

    Comparison of Three One-Question, Post-Task Usability Questionnaires (PDF, 2009)

    What is Net Promoter – Aggreko (~2010)

    Another Myth Bites The Dust: How Apple Listens To Its Customers (2011-08-26, highlights)

    How Tech Providers Improve Consumer Brand Image (2013-10-11)

    Apple among the highest in customer loyalty in 2014 Satmetrix Net Promotor Benchmarks - Apple Daily Report (2014-03-05)
     

    Attached Files:

  2. grahamperrin thread starter macrumors 601

    grahamperrin

    Joined:
    Jun 8, 2007
    #2
    Variance in NPS scores for OS X Yosemite

    The score for Yosemite has improved
    from -17
    to -13 (details below).

    Measured a week ago, with one hundred votes cast, the score was -15.

    I have not measured a variance but I'm almost certain that whenever I glanced at the figures, probably once a day or more, the score was negative.

    Seventy-seven detractors at present. I wonder how many of those people voted without trying the released build of OS X 10.10 …

    Current percentages in the poll for Yosemite

    Percentage of promoters
    31.21 + 4.46
    = 35.67

    Percentage of detractors
    4.46 + 5.73 + 1.27 + 4.46 + 1.91 + 7.64 + 23.57
    = 49.04

    NPS
    35.67 - 49.04
    = -13.37
     
  3. simonsi macrumors 601

    simonsi

    Joined:
    Jan 3, 2014
    Location:
    Auckland
    #3
    BTW you need to resist the urge to calculate to 2 decimal places, as my Math teacher would have said "just because you put 2 decimal places on it, it doesn't stop it being a guess!" (after he had read my homework :) ).

    But my point is your sample size doesn't support % to 2dp so it is a bit misleading to imply that level of accuracy ;)
     
  4. grahamperrin thread starter macrumors 601

    grahamperrin

    Joined:
    Jun 8, 2007
    #4
    There was no intention to mislead – all of the Net Promoter scores are given as integers.

    The percentages were exactly as presented by the forum software.
     
  5. simonsi macrumors 601

    simonsi

    Joined:
    Jan 3, 2014
    Location:
    Auckland
    #5
    2dp

    I'm sure you don't intend to mislead but forum software-provided or not, that result is not accurate to 2dp as it suggests.
     
  6. grahamperrin thread starter macrumors 601

    grahamperrin

    Joined:
    Jun 8, 2007
    #6
    My concern – the state of Apple software – and a plea for better measurement

    Simon, those figures were copied from the Paper Tape window of Calculator.app in Mavericks. I guess I don't know how to make best use of the app; I'm neither a mathematician nor a statistician.

    Whilst I'm not disinterested in 2dp accuracy, I am more interested in http://forums.macrumors.com/showthread.php?t=1818418 – because, I guess, there's a bias amongst MacRumors Forums voters that's not representative of Apple customers in general. But in that poll about optimism, there are too few votes.

    Keywords from the opening post here: unscientific, entertaining :D

    Joking aside: underlying all of the above, I'm deeply concerned about the recent state of Apple software – in particular, the design and UX of OS X Yosemite. If the plea for better measurements is acted upon, then maybe discussions within Apple will lead to software products that are less ridiculous.
     
  7. simonsi macrumors 601

    simonsi

    Joined:
    Jan 3, 2014
    Location:
    Auckland
    #7
    I think the point being missed here is that Apple know full well that UI changes delight and disgust users in equal measure irrespective of the change. So there is no point in trying to please everyone, it is an impossible aim to achieve.

    However the changes do generate interest and discussion but don't think Apple will ever be aiming for some kind of "zero UI impact" state.

    Functional issues are different but your describing the UI as "ridiculous" isn't an issue I can see Apple remotely being concerned about, for every individual who holds that impression, there will be another who is delighted by it - and maybe 3 potential users who look at it as a result of the publicity....1 of whom might become an Apple customer...
     
  8. grahamperrin thread starter macrumors 601

    grahamperrin

    Joined:
    Jun 8, 2007
    #8
    The subject here: NPS – apparently dismal loyalties to recent Apple operating systems

    With respect: that's neither the point nor the aim. This topic is primarily about the NPS measurements; about the apparently dismal loyalties towards recent operating systems from Apple.

    Consider the possibility that – without the constraints/bias of MacRumors Forums – the scores for iOS 8.x and for OS X Yosemite are worse than the -3% for Apple in 2013.

    Apple suggests that with the design of Yosemite, the entire experience is enhanced. That public page addresses all Apple customers, without exception. A wide audience.

    NPS measurements today, from the polls in MacRumors Forums:

    OS X Mavericks: 45 (38 respondents)
    iOS 8.x: -12 (147 respondents)
    OS X Yosemite: -12 (164 respondents)​

    A drop in loyalty, between Mavericks and Yosemite, can not be harmonious with an entirely enhanced experience for Apple's intended audience.

    From the Paper Tape window of Calculator.app for Mavericks, iOS 8.x and Yosemite:

    47.37 + 21.05
    = 68.42

    2.63 + 0 + 2.63 + 0 + 0 + 0 + 18.42
    = 23.68

    68.42 - 23.68
    = 44.74

    ----

    33.33 + 4.08
    = 37.41

    2.72 + 6.12 + 5.44 + 3.4 + 5.44 + 1.36 + 25.17
    = 49.65

    37.41 - 49.65
    = -12.24

    ----

    31.71 + 4.88
    = 36.59

    4.27 + 5.49 + 1.22 + 4.27 + 1.83 + 7.93 + 23.78
    = 48.79

    36.59 - 48.79
    = -12.2
     
  9. simonsi macrumors 601

    simonsi

    Joined:
    Jan 3, 2014
    Location:
    Auckland
    #9
    LOL - these polls aren't statistically valid in themselves, you certainly can't draw any conclusions between them, there is simply no comparable baseline between them. This is anecdotal at best however remember the golden rule "anecdotes are not data".

    I'll leave you to it.
     
  10. grahamperrin thread starter macrumors 601

    grahamperrin

    Joined:
    Jun 8, 2007
    #10
    Again, a plea for better measurement

    Understood. That's why I made the plea for better measurement, a week ago.

    simonsi, instead of laughing and abandoning the subject, could you please do something to encourage better measurement? Thanks.
     
  11. thejadedmonkey macrumors 604

    thejadedmonkey

    Joined:
    May 28, 2005
    Location:
    Pa
    #11
    XP was "new new new" when it came out. I remember it was all over the torrents, and very popular at my school.

    Vista was not popular because it ran like **** on low end computers. But the chief complaint was that they couldn't use the OS, not that they didn't like using it when they could use it.

    7 was perfect in every way imaginable.

    8 was all sorts of negative because people don't like metro, but simple tweaks like Start8 made it usable, and while you'll find people who aren't updating to Windows 8 still, even Metro haters seem to be okay with 8 if they disable the Metro part.

    And Windows 10 seems to be getting great reviews like 7 did.

    So to answer your question, I don't think even Windows has as poor of a reception as Yosemite has right now.

    My next laptop will probably be a 12" Macbook Air, but only because Apple makes cost efficient ultrabooks. The first thing I'll be doing is installing Windows on it. Yosemite is just unusable, from Safari not showing the URL to the damage it does to my eyes staring at florescent pinks and blues all day long.
     
  12. grahamperrin thread starter macrumors 601

    grahamperrin

    Joined:
    Jun 8, 2007
    #12
    Better measurements of scores will be welcome

    Yesterday, quoting a tweet from two weeks ago:

    Please tweet, post or e-mail whatever you think will be sensible … the plea is open to anyone:
    • could you please do something to encourage better measurement?
    All four topics (this one, plus the three polls) included a link to 'Net Promoter Score (NPS) Criticisms and Best Practices' – and more – in the opening posts. I never intended the results, from any of the three, to be conclusive.

    It's not enough to simply criticise the results from these polls. I genuinely did, genuinely do want better measurement – and that was not a lazy afterthought. The plea for something better was tweeted less than five minutes after the graph was tweeted.

    To anyone who would like to respond: please first read some of the items linked from the opening post. If your criticisms (of the polling method and so on) are significantly different from the criticisms already given – the 2009 blog post by Verint Systems might be a good starting point – then please, add relevant details to this discussion.

    If you must criticise, then please also do something positive and constructive towards getting better measurements of Apple customer loyalties towards the three operating systems. ​


    (Personally, I'm less interested in iOS. But it seemed fair to poll about iOS at the same time as the other systems; I didn't want to be exclusive.)

    Thanks
     
  13. grahamperrin thread starter macrumors 601

    grahamperrin

    Joined:
    Jun 8, 2007
    #13
  14. grahamperrin thread starter macrumors 601

    grahamperrin

    Joined:
    Jun 8, 2007
    #14
    Customer satisfaction/loyalty – Apple hardware, Apple software

    From an Apple operating systems design topic:

    Did that report refer to the American Customer Satisfaction Index?

    Personal computers

    Without reference to software: satisfaction with Apple dropped three percent between 2013 and 2014. The press release Desktop Customer Satisfaction Surpasses Tablets and Laptops American Customer Satisfaction Index (2014-09-23) describes PC makers so I assume that this measure of satisfaction is hardware-oriented. (Not a measure of satisfaction with OS X.)

    Cellular telephones

    Again without reference to software: satisfaction with Apple dropped 2.5 percent. I assume that this measure is iPhone-oriented. (Not a measure of satisfaction with iOS.)

    Mobile phones (NCSI-UK)

    Without reference to software: up one percent between 2013 and 2014. ​

    Shareholder influence

    I don't know about first, but there's a sense that increasingly, as a result of excessive shareholder influence, too many Apple customers are dissatisfied with changes to products. I sort of wonder what goes on at shareholder meetings (but I'm not inclined to read relevant documents – shareholding is of no interest to me).

    Influencing the influencers

    If it's possible to capture the attention of a broad range of influencers, including shareholders – to make enough people across that range realise (in simple terms) that some types of change are detrimental to the brand – then there'll be a better chance of future redesigns pleasing a greater percentage of customers.

    How to capture that range of attention? What do people love to tout? Numbers.

    [​IMG]

    – I keep coming back to that because of the wish for better measurements of loyalty towards OS X Mavericks and Yosemite (and to whoever might make those measurements, it'll probably make sense to perform measurements for iOS around the same time). Any one of the following might be within the results:
    • a simple negative correlation (for the increased version number, decreased loyalty)
    • a simple positive correlation
    • nothing remarkable …

    … if there's a negative correlation, and if that's significant enough to attract the attention of Apple and/or shareholders, it'll be for Apple to balance priorities in software design reviews. The positive aspect of this approach to negative attention could be that it's not singling out any individuals or groups as 'to blame'. As far as

    ----

    If there's a positive correlation – if, say, there's greater loyalty towards Yosemite than towards Mavericks – then that result will be a good complement to the poll about optimism in these forums.
     

Share This Page