Consumer Reports Retesting MacBook Pro Battery Life After Apple Says Safari Bug to Blame

[HobeSoundDarryl]

Seems to me Apple resolved the problem yet more whining.

No, this just altered the standardized testing for all laptops to improve Apple's results. Basically this is "you're holding it wrong" again. With that phone, holding it differently yielded better results. With this testing, changing a common testing parameter used with all laptops is going to yield a much better result...

Consumer Reports has issued its own statement on the matter to explain why it turns off Safari caching during its testing and other details:The non-profit organization also acknowledged user reports of poor battery life that have surfaced over the past three months.

Key quote: We also turn off the local caching of web pages. In our tests, we want the computer to load each web page as if it were new content from the internet, rather than resurrecting the data from its local drive. This allows us to collect consistent results across the testing of many laptops, and it also puts batteries through a tougher workout.

Of course, for this to be an objective comparison, CR would have to allow cached web pages to work with other laptops too. So, assuming other stock browsers cache page content as well as Safari, the competitors battery life will rise with this testing change too.

So conceptually, this might bump up CRs battery life review of the new MBpro. But it should also bump up all of the other laptops against which this MBpro is compared.

I guess this means Consumer Reports doesn't know what their doing

No, it means we'll grab onto anything to support Apple. Consumer Reports is GREAT at what they do... one of the very few- perhaps only- consumer product rating source that doesn't have any kind of "payola" scheme or bias to influence their reviews. They don't even take advertising dollars to avoid that apparent conflict of interest too.

Else consider the many times that CR has rated other Apple stuff best or near the very top of their ratings. Did they "not know what they were doing" there too?

 

[ScottNWDW]

For me, the 2016 MacBook Pro 13" with Touchbar and 512GB is the best MacBook Pro I've ever owned, hell it's the best Mac I've ever owned.

 

[FatherJack1980]

This is how I imagine the meeting between Apple and Consumer Reports went (Apple being the one in the brown suit and smiling)

But boy, I would have thought Consumer Reports would have default settings on everything on all laptops they test (just as about every John Doe that buys a new laptop - not that many tinker around with settings). Now it makes more sense

 

[Zorn]

I understand the desire by CR, but that isn't actual usage. If they wanted to mimic how most of us use the machine they'd not turn off the caching. While its good that apple fixed the bug, it seems kind of sketchy for CR to do that

Yeah, I completely agree. It seems really odd for them to say that they aim to test it under the same type of conditions a consumer would experience, then go out of their way to enable this hidden setting because they randomly want the browser to pull each page refresh from the server. If that isn't a setting most people will ever enable, and thus not how most consumers will use the laptop, it's pretty arbitrary.

 

[patrickbarnes]

I understand the desire by CR, but that isn't actual usage. If they wanted to mimic how most of us use the machine they'd not turn off the caching. While its good that apple fixed the bug, it seems kind of sketchy for CR to do that

It's not sketchy at all. It's the only way to test multiple different types of laptops and compare how they do.

Disabling caching in a browser should not cause this type of battery drain. It simulates real world usage of a web browser.

People are confusing two issues here. Apple found a bug with something to do with icons. That's unrelated to the cache bypass setting.

 

[Naimfan]

I understand the desire by CR, but that isn't actual usage. If they wanted to mimic how most of us use the machine they'd not turn off the caching. While its good that apple fixed the bug, it seems kind of sketchy for CR to do that

(Emphasis added.)

You're kidding, right? They test EVERY OTHER LAPTOP the same way, including prior Macs.

And, by the way, that IS actual usage for me.

 

[RedOrchestra]

If that isn't a setting most people will ever enable, and thus not how most consumers will use the laptop, it's pretty arbitrary.

No, that's what Phil the Apple Shill said ... sweeping, sweeping, sweeping ...

 

[maflynn]

(Emphasis added.)

You're kidding, right? They test EVERY OTHER LAPTOP the same way, including prior Macs.

And, by the way, that IS actual usage for me.

No, I'm serious, I think you get a better result set if you set out to test a product the way its supposed to be used. I understand they didn't single out the Mac on this, but that doesn't mean I think its not a good way to test.

Just my $.02

 

[69Mustang]

No, I was just stating my opinion in a developing conversation rather than making a pointless reply/quote such as you.

Calm down Susan. It was a joke about refreshing the cache, which was why I bolded it. Nothing about your opinion

 

[stevet]

Clearly not a Pro machine if you are required to use Safari to get best battery life.
The older MBP laptops could use any browser, or amazingly it it could run Pro tools like Adobe products without killing the battery. Apple, if you say it has up to 10 hours for a Pro laptop, then we expect around 8 or 9, not 2 or 3. Safari is no excuse.

This is straight BS. Run Chrome on any Mac LapTop and you will see it burns more battery life than Safari.

 

[magicman32]

So, Apple was right. That is the bottom line. Man, this place has become negative beyond belief.

Negative? That can't be. You have certain people, one posted in this thread, who believe this is a place of magical unicorns where everyone loves and defends everything Apple does.

So be careful saying Apple was right. You might get ostracized for being an Apple "fanboy" despite the negative posts outweighing the positive by a large margin. Their alternate-reality narrative can't bear the weight of such logic.

 

[satchmo]

Do people still turn to Consumer Reports? It's certainly not a website or magazine I think of when looking to buy a MacBook Pro.

With the plethora of popular tech sites out there, I'm surprised Apple paid much attention to them. That said, the executives at Apple all probably grew up reading CR.

 

[kognos]

For a company called "Consumer Reports" they weren't testing products in the way Consumers would.

I don't know but this whole thing smells like a goofup on CR end.

Sure there might be a but in Safari, but it's kind of like testing a car and saying "it doesn't get 0-60 in 10s, it doesn't move at all!" and then later finding out that they removed the wheels prior to testing. Consumers wouldn't do that.

Testing methodology was the blame here, maybe 5% of that blame should go to Apple for a Safari glitch but still, that's weak.

 

[AdeFowler]

I have to say that my battery life has been nothing but amazing. I was coming from a 2012 MacBook Pro, but the battery life is incredible.

 

[jwm]

Wait.... Did Apple just admit there is a bug in Safari?

 

[HobeSoundDarryl]

Did you think for 5 seconds before you posted this?

Of course this could have been avoided if CR reached out to Apple before rushing to publish. Something is clearly wrong when their test results had such wide variances.

One corporation deserves special exception when a review is coming out bad?

Or are you saying every time CR has a less-than-stellar review for ALL of the products they review, they should reach out to the manufacturers and seek ways to improve the ratings?

It's CONSUMER Reports, not CORPORATION reports or APPLE INC Reports. They put us consumers first... above the corporations. And they just call 'em as they see 'em. THEY did not put this bug in Safari. And, presumably, they've used the same testing parameters in prior Apple hardware tests, which generally led to stellar ratings vs. other products.

 

[a.coward]

The real questions:

1) When performing tests on other laptops, do they disable the cache also?
2) What happens if you do as recommended by Apple and re-enable the cache, but use dynamic web pages in the test.
3) What happens when you use another web browser in the tests which cache disabled -- Firefox or Chrome.

Anyway, I do think that they are "testing it wrong", especially if they are turning off the cache. Setup your test environment correctly so that the content you browse is dynamic, and test using all default settings in the browser/OS. However, I'm really not about to take Apple's word on this one as it almost appears they are trying to influence the test environment (i.e. make sure you load the same web page so that it never has to access the internet -- but saying it with a marketing twist). That said, there are also enough reports of poor battery life to indicate are still real issues with this line of laptops.

 

[RedOrchestra]

Of course this could have been avoided if CR reached out to Apple before rushing to publish. Something is clearly wrong when their test results had such wide variances.

What a pathetic response to cover up for Apple and the anointed Chief Shill Phil and an acknowledged bug in Safari.

 

[69Mustang]

No, I'm serious, I think you get a better result set if you set out to test a product the way its supposed to be used. I understand they didn't single out the Mac on this, but that doesn't mean I think its not a good way to test.

Just my $.02

Here's why they turned of caching: "For the battery test, we download a series of 10 web pages sequentially, starting with the battery fully charged, and ending when the laptop shuts down. The web pages are stored on a server in our lab, and transmitted over a WiFi network set up specifically for this purpose. We conduct our battery tests using the computer’s default browser—Safari, in the case of the MacBook Pro laptops"

The problem wasn't the test. The problem was a bug in Safari when caching was disabled. Every previous macbook passed the test. Safe to assume none of the previous macbooks had that Safari bug.

 

[HobeSoundDarryl]

Consumer Reports' big failing wasn't so much the test conditions, which seemed reasonable, but in accepting such wildly varying results. It's clear they wanted to publish a click-bait article and didn't care about investigating the issue any further.

CR doesn't sell advertising. How are they making money from click baiting?

 

[Fancuku]

A bug in Safari. Yeah, ok.

 

[ArcaneDevice]

Yeah, I completely agree. It seems really odd for them to say that they aim to test it under the same type of conditions a consumer would experience, then go out of their way to enable this hidden setting because they randomly want the browser to pull each page refresh from the server. If that isn't a setting most people will ever enable, and thus not how most consumers will use the laptop, it's pretty arbitrary.

This is a Macbook Pro. So if it was used in a capacity that it's intended user base would use then it would have the developer mode enabled in Safari. I use it all the time. Not only because it's helpful for design professionals but because it reveals a lot of useful tools that any power user would want.

The fact that this device is apparently not intended to be used in dev mode indicates that Apple once again don't have any interest in what professionals actually do.

And to cap it all, Consumer Reports easily found a bug that affects actual professionals for Apple because Apple couldn't be bothered.

 

[belvdr]

"This is the best pro notebook we've ever made..."

Let me guess...Schiller crafted this line? Marketing spin 101.

No Apple crafted that line. It's been used by several Apple folk; maybe it's part of the onboarding process to state things like this.

"This is the best meeting we ever had."
"This is the best campus we ever built."
and so on...

 

[neuropsychguy]

Maybe they don't actually take issue with these things like many of us who purchased this laptop.

You mean like a few people on MR and other websites. Many commenters online forget that they do not represent the general consumers of these devices. Yes, some are "pro" users that have issues but most people do not. Take the removal of the headphone jack from the iPhone 7. I personally know 10 people with iPhone 7 devices (myself included - only 3 of us are "techie"). Not one has complained about no headphone jack. A couple people commented on it but said "I'll just use an adapter or I'll use the included headphones". That's anecdotal evidence but things like this really are not a big deal to the majority of people. Most people don't care about tech - they use it but aren't excited by it. The same goes for the new MacBook Pro. Most people just use it and don't care if they have to buy a dongle or if they get a little less or a little more battery life. People buy the MacBook Pro, turn it on, it works well, and they go to work.

 

[Martyimac]

So CR tested using settings that are available to all users, yet the MBP defenders are saying they did it wrong? But Apple comes along and finds a previously unknown bug, lets the world know that CR was right, and folks are piling on CR?

Head shake, you can't have it both ways folks.