Consumer Reports Retesting MacBook Pro Battery Life After Apple Says Safari Bug to Blame

[HobeSoundDarryl]

I'm not a US citizen and hadn't heard of Consumer Reports before; but from the previous article here on Macrumors, I didn't read anything how they are testing one product and then comparing it to all others; what my understanding was from that previous article was that they test products to see if it lives up to the claims of whoever manufactures them

CR main business is to review many products head-to-head. They typically present these in big tables of results. For example, here's one of their sample review pages comparing consumer credit cards:

They will write articles about new product releases- especially for very popular or hotly anticipated products. Why? Because consumers who trust CR want to know what they have to say before laying out the money for a new product.

And it really doesn't matter if you are a U.S. Citizen or not. CR reviews should be applicable to consumers everywhere. Products are generally products. Their head-to-head reviews should help you make good decisions on anything you might want to buy (that they review).

 

[manu chao]

My daily use is modest (web browsers, WiFi, and text editors), and my results were extremely consistent: ~4.5 hours of battery life. My 2013 retina MBP 15 was consistently good for 7-8 hours performing the exact same tasks using the same software.

And you really think that reducing the battery size from 95 Wh (2013 15" MBP) to 76 Wh (2016 15" MBP), ie, a reduction by 20%, is responsible for a reduction in the battery life of 40% (7.5 to 4.5 hours)? And if only they had kept the battery size constant, you would have gotten the same battery life? Or do you think that the Touch Bar is consuming about 20% of the battery (with your usage pattern)?

 

[TMRJIJ]

The bigger story here is that Apple's software appears buggy as hell. They need to do a couple snow leopard like releases where they just focus on cleaning up garbage code instead of pumping out new features.

Sorry, I don't really feel like losing 500 GB of data again just because the Guest Account is on.

 

[djcerla]

As widely expected, CR methodology was completely screwed.

But they got their annual page views bump, so success!

 

[az431]

Definitely the best MacBook Pro I have owned. Not saying the last generations weren't good either, but these are great machines. Wonder when that Safari fix will make it to us.

There is no "fix". Consumer Reports disabled browser cache in the Develop tab, which forced constant reloading that consumed more power. Most people don't know how to access that setting, nor would they have a reason to disable cache if they knew how.

 

[69Mustang]

It isn't about that. It's about a bug that needed to be fixed. The testing protocol uncovered a bug, and that bug lead to poor battery life. End of story.

I agree. That should end the story

Consumer Reports needs clicks. They need to post the most terrible, ridicule inducing reports they possibly can because that controversy leads to page clicks. That's what motivates them.

Why do you think CR needs clicks?

 

[az431]

The classic Apple "You were testing it wrong" ploy

Try reading the entire article from beginning to end, instead of just skimming the title.

 

[HobeSoundDarryl]

You posted a boatload of stuff unrelated to the comment I made. You said "No, this just altered the standardized testing for all laptops to improve Apple's results."

It isn't about that. It's about a bug that needed to be fixed. The testing protocol uncovered a bug, and that bug lead to poor battery life. End of story.

It is NOT about Apple wanting to be tested differently than everyone else, though I'm sure that all manufacturers would like the testing protocol changed because it would improve everyone's results, not just Apple.

Consumer Reports needs clicks. They need to post the most terrible, ridicule inducing reports they possibly can because that controversy leads to page clicks. That's what motivates them. I used to rely on them for things like dishwashers, but I've since found that to be a mistake as well. Best to just talk with people that repair all of them.

Again, I stand by what I wrote.

And, it's not CR's job to find bugs in software. That one turned up is not CR's fault.

And, why do they need clicks? They don't sell ads. They're no particular revenue benefit from any click-baiting tactics. They ONLY make money from subscription sales and the meat of that article was freely summarized for all who wanted to see it (no subscription required).

Make up what you want but I'm guessing you just don't understand CR. They don't shill for anyone. The vast majority of their reviews of Apple hardware have been very, very positive. How can they be so wrong only when they find some fault with ONE Apple product?

 

[smaffei]

so Safari kills battery life and Chrome kills battery life. Basically we shouldn't browse on our new Macs =/

Exactly! If a pretty typical software defect drains the life of the battery that badly, then your computer is far from robust. Definitely not professional level (because intensive tasks would drain the battery like that too). Eh, Apple?

 

[dannys1]

Plot twist: This is his first MacBook Pro.

As you can see from my signature, I buy a lot of Apple stuff and I've had 3 refreshes of the retina MacBook Pro including the 2015 and this blows it out of the water (even though i've had build quality issues) for me, its macOS bugs that are causing me issues, the hardware is fantastic and the most enjoyable MacBook i've ever used.

 

[Exile714]

CR doesn't sell advertising. How are they making money from click baiting?

The end goal of click baiting isn't necessarily ad revenue, but increased traffic. In CR's case, their relevance is dwindling and bringing more people to their website (and more people generally talking about them, as we are right now) helps remedy their relevance issue.

Sure, it won't lead to ad revenue or even necessarily subscribers, but getting more eyes on their site is worth putting out a controversial test result.

That's not to say they've 100% figured out all their modern issues yet...

https://www.google.com/amp/www.theatlantic.com/amp/article/477108/?client=safari

 

[t0mat0]

So how did they have the longer than possible results? (better than just leaving the MBP on with the screen on)?

 

[racer1441]

What's the test needed to show Consumer Reports is a bunch of has beens or never were 'scientists' or just actors wearing lab coats? I'm in for that.

 

[d0nK]

I have to think that if this was Microsoft this wouldn't be happening.

If this was Microsoft, the same-specced machine would cost about £1000 less.

 

[dannys1]

so Safari kills battery life and Chrome kills battery life. Basically we shouldn't browse on our new Macs =/

How on earth have you come to that conclusion from the article??
[doublepost=1484075567][/doublepost]

If this was Microsoft, the same-specced machine would cost about £1000 less.

Really? Have you see the price of the Surface Book??
[doublepost=1484075620][/doublepost]

So how did they have the longer than possible results? (better than just leaving the MBP on with the screen on)?

And that is the bigger question - they got those with Chrome they record too. I can't help think they have no idea what they're doing - get 17 hours from it is stupid.

 

[satchmo]

Given the user base of Apple is as generic and mainstream as it gets now Consumer Reports is perfectly at home testing Apple products next to refrigerators and is still a well known brand and central focus for product reviews.

You think the majority of Apple customers are going on the internet checking out Geekbench and getting their reviews from Ars?

They may not turn to Geekbench or Ars, but they certainly not visiting Consumer Reports either.

Plus I wouldn't underestimate the mainstream. I think the majority of people would simply do a quick search for 'MacBook review'. When you do, you'll get the likes of Forbes, NY Times, TrustedReviews, cnet, MacWorld, Engadget, Wired...etc. Plus there are tons of user generated review on YouTube.

Some are tech focussed, but many are mainstream publications. And unlike Consumer Reports, the reviews are subscription FREE.

 

[TMRJIJ]

Exactly! If a pretty typical software defect drains the life of the battery that badly, then your computer is far from robust. Definitely not professional level (because intensive tasks would drain the battery like that too). Eh, Apple?

You don't understand coding that much, do you?

 

[nwcs]

Man, so many people here have no idea what it would take to be a software or hardware tester... What CR did was absolutely the right approach. Note, I'm not saying that the results matter just that it was the right approach. If you're enabling caching and such you're no longer even coming close to testing a repeatable dataset. You're testing the caching approaches only.

People think the CR test was intended to be a real user test. There is no such thing. How could that even be defined? There is no typical user and there is no "intended usage" from Apple. Without having the MBP in hand for your very specific tasks there isn't a way to know how long you would have in battery life. What the CR test does is provide a level playing field for comparing against other laptops. That's it.

 

[HobeSoundDarryl]

The end goal of click baiting isn't necessarily ad revenue, but increased traffic. In CR's case, their relevance is dwindling and bringing more people to their website (and more people generally talking about them, as we are right now) helps remedy their relevance issue.

Sure, it won't lead to ad revenue or even necessarily subscribers, but getting more eyes on their site is worth putting out a controversial test result.

That's not to say they've 100% figured out all their modern issues yet...

https://www.google.com/amp/www.theatlantic.com/amp/article/477108/?client=safari

Very well. So some kind of marketing conspiracy theory. For years and years, CR has generally been too dumb to put out a negative Apple result on almost everything else Apple makes so they can get this free publicity, eyeballs and perhaps some more subscribers to boot? However, they finally figured out that compromising a many decades reputation of ratings objectivity with some negative bias against this one product is THE way to "remedy their relevance issue." Sure, let's try to buy that.

 

[lordofthereef]

Safari is made by Apple, it isnt third part software.

Right. See what the person I quoted was saying...

The implication was you "have to use safari for best battery life" and thereby it's not a pro machine. Of course Apple isn't going to test third party browsers. You don't "have to" use safari. But the numbers they advertise will be based on their safari testing....

 

[dwaite]

I think the main question is what is this icon bug about? Does it cause 100% CPU and battery drain? Need more detail on that to know.

If you think what disabling all caching would mean, scrolling a page or navigating away and back to a tab would require the page with images, css, etc to reload.

So the behavior CR probably intended was to have the browser behave as if the cache is cleared before each request. They may have assumed thats what Disable Cache did, and it may have had that behavior in previous versions of Safari due to the view being preserved longer.

However, what it sounds like they actually got with current Safari was the laptop having to repeatedly fetch/parse/render some of the content - because those developer settings are not meant as part of normal usage and aren't normally tested for efficiency/battery life.

I tend to side with people saying this was incredibly click-baity behavior by CR- they didn't need to draw any attention whatsoever publicly that they weren't giving a recommendation to the laptop (they have plenty of laptops in their ratings which they don't give the coveted recommendation to), but chose to make a big deal out of it for the attention.

 

[coolfactor]

Clearly not a Pro machine if you are required to use Safari to get best battery life.
The older MBP laptops could use any browser, or amazingly it it could run Pro tools like Adobe products without killing the battery. Apple, if you say it has up to 10 hours for a Pro laptop, then we expect around 8 or 9, not 2 or 3. Safari is no excuse.

EDIT: Could we stop ranting about how Chrome is better. I'm just trying to point out that Apple is no longer designing this laptop to have good power for Pro software like Adobe products. Chrome I would not consider a Pro product.

You're way off base here.

While Safari remains my preferred and default browser, Chrome easily outpaces it in recent versions in terms of memory usage, stability, and responsiveness. It's the preferred browser by many. Did you know that Chrome was born from the same WebKit project that Safari is built from? Google has since forked the project and Chrome is now running an very different codebase. This gave them the freedom to implement their own ideas and optimizations, and the results are showing.

As a website developer, I use Safari, Chrome and Firefox side-by-side all day long. Similar to how one would use Microsoft Word and Excel side-by-side for different purposes, that's how I use browsers — to access specific web applications to do my job. Safari is set as my default, but frequently locks up when I have many tabs open. Firefox freezes after long periods of having a Google Doc open, but Chrome keeps on ticking.

You started out by saying the opposite of what this article is about. The findings were that using Chrome exhibited better battery life — with caching disabled — but that's only because of a Safari bug that Apple has since fixed. Now both browsers operate equally well. And both have very advanced developer tools. Chrome is at least equal to Safari in terms of professional capabilities.

 

[Weaselboy]

However, if using the cache becomes the new protocol, all other laptop results are likely to rise too (because they'll get the very same benefit of working from cached content).

Not likely. It was not the disabling of the cache that caused the bad battery life. It was an icon caching bug in Safari that only manifests if the cache is disabled that caused the problem. So this is not likely to change test results in anything except this specific macOS/Safari version with the bug.

 

[WhatUp?]

It still doesn't fix the problem with the 2016 model having 1/3 smaller battery than the 2015 model.

My 2015 MacBook pro battery life sucks since I updated the OS.

 

[JanDdd]

Exactly! If a pretty typical software defect drains the life of the battery that badly, then your computer is far from robust. Definitely not professional level (because intensive tasks would drain the battery like that too). Eh, Apple?

It is fairly simple. If your computer has a 45 watt tdp cpu and a 75 Wh battery don't expect it to last 2 hours on the heaviest of workloads. No Macbook Pro has ever lasted more than a mere few hours under full load. That kind of math applies to any laptop...