Consumer Reports Retesting MacBook Pro Battery Life After Apple Says Safari Bug to Blame

[JungleNYC]

Definitely the best MacBook Pro I have owned. Not saying the last generations weren't good either, but these are great machines. Wonder when that Safari fix will make it to us.

I agree. While I was NOT psyched for this machine after waiting since 2008 (thank you, thank you)…it's been better than I imagined. Time will tell, of course, but I think it's a very similar upgrade to the current iPhone: not much significant design improvements, but rather tighter, zippier, more colorful version of the previous models. I've been pleasantly surprised so far. THAT SAID I agree that this probably shouldn't be called "pro."

Although as a "consumer" I do cheer-on the BitchFest that goes on, because I do think Apple has gotten a bit lazy lately.

 

[2IS]

Well, it doesn't really speak for Consumer Reports that they didn't do additional testing. If it's a Safari setting, then an alternate battery life test with Chrome (very common use case) would've easily showed the anomaly in battery life. Why do they have so much credit anyway?

Probably because their sub par battery life findings mimicked what other users were experiencing.

 

[jerryk]

The big question is if this was the same setting they used to test prior MacBook Pros. If so, then they must use this setting or else you are not comparing like to like, or write a disclaimer that says the testing methodology has changed.
[doublepost=1484090711][/doublepost]

I understand the desire by CR, but that isn't actual usage. If they wanted to mimic how most of us use the machine they'd not turn off the caching. While its good that apple fixed the bug, it seems kind of sketchy for CR to do that

But sites do this all the time to create interactivity. When you load the page you dump the cache so any controls reloaded anew. Also you may have a timer running to reload the page's content with the latest information so the displayed data is not stale.

 

[KingOfStuff]

I'm no Apple apologist, but the Consumer Reports test was laughably amateurish, as even the author admitted in the article that Safari could be to blame.

An amateurish test tossed a computer costing $3.5k.

 

[oldmacs]

Oh is this Safari fix going to fix the fact that Apple prioritised thinness over battery life? Amazing.

 

[kdarling]

Why do they have so much credit anyway?

Consumer Reports is a sixty year old NON-PROFIT consumer testing magazine.

Unlike most of today's internet-only click-bait oriented news sites, they do not accept advertising.

They buy everything they test. No freebies from companies looking for publicity or favors.

All that's why people trust their test results to be pretty unbiased. (Some take issue with their long term car tests, but they're still a good starting point for research.)

 

[MrGuder]

What interesting about this is 9 to 5 mac ran an article about how something was off with Consumer Reports testing and Consumer Reports actually responded to 9 to 5 mac by saying

"In this case, we don’t believe re-running the tests are warranted for several reasons. First, as we point out in our original article, experiencing very high battery life on MacBooks is not unusual for us – in fact we had a model in our comparative tests that got 19 hours. Second, we confirmed our brightness with three different meters, so we feel confident in our findings using this equipment. Finally, we monitor our tests very closely. There is an entry logged every minute, so we know from these entries that the app worked correctly."

I'm sure Apple made a call to Consumer Reports and after that call, Consumer Reports changed it's mind. lol

 

[KingOfStuff]

It's not a ploy if Apple was right.

But you don't think CR was testing it wrong? Do you disable all browser caching? Does anyone? This really is another CR fail. What's the point of testing battery life if your results are irrlelcant to how people use their computers?

I betcha I could test a car and I've the engine "a tougher workout" by driving on the highway in second gear. Not sure why anyone should conclude that the engine's faulty.

that doesnt make sense. these options are there, what few we have in OSX, i shouldnt have to be weary of which ones i use because it's not how other people use their computers.

 

[Black Magic]

I guess this means Consumer Reports doesn't know what their doing. lol ok

They don't know what they are doing. The crazy part is that they charge consumers access to their content and these consumers use the info on making purchasing decisions. Imagine how many other products they are testing in some obscure manner giving out misinformation?

I think all review sites should be transparent and post all details about the test upfront. This will allow for others to replicate the test and confirm results if ever questioned. This will also show if the test reflects real world usage.

 

[RangerOne]

As usual, few people commenting here actually read the article:

#1: The browser was configured wrong, to disable cache so it constantly has to re-transfer data from the server. Even as a developer, I almost never use this feature. Why CR decided to set this is beyond me.

#2: The little-used feature illustrated a bug which caused icons to continuously download...because they weren't cached. So this escaped Apple's QA, but an easy software fix.

I've owned just about every Mac since 1989. There are a few minor nits with the late-2016 MBP15 machine I'm now typing this on, but I have no question that it's a fantastic machine. Sure, the keyboard is a little louder (but I find myself typing faster). Yes, some people wanted the new Intel processors (but that would have meant waiting at least half a year more). Yes, the lack of ports and MagSafe are a little irritating (but the whole world is moving to USB-C). Lower battery life under heavy load while sitting in a coffee shop editing 4k video or playing graphics-intensive games might be an issue for some (but just plug in - problem solved).

As an open-minded power user, I absolutely love my new machine and wouldn't give it up for the world. Nothing is ever perfect, but easy workarounds exist.

 

[Zskyline]

Such bs. The MacBook Pro only gets 3 1/2-4 hrs at best (real world) no matter what you do. It's not a "Safari Bug" Its a design flaw. Apple lets Jony Ive make expensive art, not good laptops anymore

 

[kdarling]

#1: The browser was configured wrong, to disable cache so it constantly has to re-transfer data from the server.

Why do people keep repeating this nonsense.

It was NOT configured wrong for a test like this, where the desired action is that you want the data to constantly retransfer, so you get a consistent load.

Not even Apple said it was wrong to change that setting. They merely said, in order to reassure users, that it is not a usual setting.

The ONLY thing that was actually wrong, was the Safari icon fetch code, and now that's fixed.

 

[wozmatic]

Another reason why:

- Apple needs to test their products more in-depth
- Safari is an inferior browser and should be scraped
- Stop charging people $4000 for a laptop

#DellXPS15

 

[Abazigal]

Having a bug screwed Apple by handing themselves some negative press. CR really had nothing to do with it at all, really; they were just the messenger.

And they delivered the wrong message, which makes them a fairly inept messenger.

 

[RedOrchestra]

Why do people keep repeating this nonsense.

'Cause they got nothing else to hang onto.

 

[Greymacuser]

The classic Apple "You were testing it wrong" ploy

Yep. Sounds like what a child would say when caught messing up on their homework.

 

[GSDLVR123]

It doesn't deliver the advertised battery life by a large margin.

I have run copious real usage tests. It under performs advertised life by around 30 percent and has 10 percent less battery life than then 2015 model.

I've been following this 'issue' since Christmas. I purchased 2 13" touch bar MBP Laptops (16GB) for my young kids this Christmas. Last Christmas I purchased a 2015 rMBP 13" (with I7 and 16GB) for my oldest daughter. Since Christmas I've had them run countless tests comparing the 3 machines. As of today the touch bar machines get 17-22% less battery life than the 2015 model. The Apple Care folks have been very patient with me - as I AM NOT a computer expert but just a concerned dad running my own real world tests. They did tell me - before I started my daily tests - to expect 20% less battery life on the touch bar machines. I have to say - for our situation that is not a big deal. I must say Apple offered to either refund or exchange my machines - but my kids like them and my daughter is happy with her 2015 machine.

 

[wallysb01]

And they delivered the wrong message, which makes them a fairly inept messenger.

It is a bug that effected battery life. CR basically guesses the exact problem in their original article. Its not their job to figure out why, fix it and rerun tests. CR will rerun tests when the new version of macOS is released, which IIRC they also stated they would do before this new Apple release.

 

[Count Blah]

[doublepost=1484099319][/doublepost]

I understand the desire by CR, but that isn't actual usage. If they wanted to mimic how most of us use the machine they'd not turn off the caching. While its good that apple fixed the bug, it seems kind of sketchy for CR to do that

If you cannot see the intent of their method, then I don't know what to say. Perhaps you could provide 100k random pages for them to hit instead. Otherwise, having a set of known good sites, and preventing caching, seems like a logical method for FORCING safari to treat each page load, as mostly unique.

It's not CRs fault that Apple has issues with safari. Given CRs use of chrome as a control, they pretty much say that safari is likely to blame.

It's almost as if you are upset that CR found a Safari bug that Zaps battery.

 

[unplugme71]

What's odd is that all 3 were different with the bug, but without the bug, they all work very similar.

Also, CR should've mentioned the port issue, smaller battery size, etc and gotten Apple to rethink their ****.

Happy to see this is resolved and we can end the battery test fiasco.

 

[Tech198]

Makes sense, u don't wanna test performance with caching on : negative results.

Caching of anything, reads on a SSD, memory checking, website loading etc "improves" performance. Its caches stuff for faster re-loading times of websites, opening of apps from SSD, etc.. and recently used. and is is not actually testing true performance. For this to be anything accurate u must disable caching.

With web site loading, u can override the cache by keyboard short cut (both Safari & Firefox)

 

[Juan007]

Consumer Reports is a DYING stone-age publication that is no longer relevant. Toilet paper and tampon reviews no longer move the needle. Another "MacBook Pro is the best laptop ever" article gets lost in a sea of 500 more. The only way they can draw eyeballs is to make up controversy, just like they did on iPhone 4. Steve Jobs didn't mention them once, he knew they were irrelevant trolls trying to dig up dirt on a near-perfect product. BIG MISTAKE by current Apple management to even acknowledge these fools.

Apple, keep being great, ignore the haters.

 

[dogslobber]

We're lucky that CR is there to hold Apple accountable as nobody else does.
[doublepost=1484102189][/doublepost]

"We also turn off the local caching of web pages. In our tests, we want the computer to load each web page as if it were new content from the internet, rather than resurrecting the data from its local drive. This allows us to collect consistent results across the testing of many laptops, and it also puts batteries through a tougher workout.

According to Apple, this last part of our testing is what triggered a bug in the company’s Safari browser. Indeed, when we turned the caching function back on as part of the research we did after publishing our initial findings, the three MacBooks we’d originally tested had consistently high battery life results."
​

Maybe if Apple did their own QA instead of expecting third party testers to do it then this wouldn't happen. Yes?

 

[Abazigal]

[doublepost=1484099319][/doublepost]
If you cannot see the intent of their method, then I don't know what to say. Perhaps you could provide 100k random pages for them to hit instead. Otherwise, having a set of known good sites, and preventing caching, seems like a logical method for FORCING safari to treat each page load, as mostly unique.

It's not CRs fault that Apple has issues with safari. Given CRs use of chrome as a control, they pretty much say that safari is likely to blame.

It's almost as if you are upset that CR found a Safari bug that Zaps battery.

It's a bug that simply won't apply to the majority of MacBook Pro users who won't run their browsers in that way ever. So maybe Consumer Reports should rename themselves to "0.1% of consumers who use their computers in the same idiosyncratic way we do" Reports.

You are right - it sounds logical, except it wasn't. They assumed Safari would work in a certain manner (except that Apple wasn't obligated to make Safari work the run they felt it ought to run). So the fault is on CR here for making an erroneous assumption, not bothering to verifying their hypothesis, and running with it.

 

[Onexy]

As usual, few people commenting here actually read the article:

#1: The browser was configured wrong, to disable cache so it constantly has to re-transfer data from the server. Even as a developer, I almost never use this feature. Why CR decided to set this is beyond me.

If you'd be a developer you know you'd rather not hardcode thousands of different websites urls for a test. You'd rather reload a couple of websites again and again. And you'd know that due to power saving/cache features reloading the exact same URL would put too little stress on the machine to be realistic usage.