Consumer Reports Retesting MacBook Pro Battery Life After Apple Says Safari Bug to Blame

[Weaselboy]

I'm not accusing CR of click baiting because I don't know one way or the other, but some of the comments here make it sound a bit like CR is some altruistic organization because they don't accept ads.

CR is there to make money. If you go to the review in question here you will see you cannot even read the review unless you buy a CR subscription. So they do have an interest in driving traffic to their reviews.

 

[macsrcool1234]

And had CR reached out to Apple it would have been discovered before they published. Again it wasn't that the battery results were bad it's that they were incredibly inconsistent. Wouldn't normal course of business cause someone to analyze results before publishing them? And if the results are that abnormal perhaps do a bit more analysis before publishing? Seems clear to me CR rushed to publish this to make headlines. Every tech site out there reported on their findings.

No.

CR does not work for Apple and they are under no obligation to 'wait' for them to do anything. Apple doesn't get special treatment. Apple found and fixed a bug because of their prompt attention.

 

[brentsg]

Now back to Apple. I point out yet again that the vast majority of CR reviews for most of Apple's other stuff have been generally stellar or better. Where were you to call them garbage reviews when they were praising Apple stuff?

I likely didn't post because I didn't and don't read their reviews. I haven't subscribed in many years. I regret reading about this one because it simply results in a lot of back and forth argument and I honestly have little interest in.

I'll go back to working on my 2010 Mac Pro that Apple doesn't want me to replace.

 

[sudo1996]

What kind of bug gets triggered by disabling the cache? They should make sure it performs well when actually downloading pages rather than using the cache!

 

[montanaco]

I want to be happy Apple found a bug and quickly fixed it but MacRumors is telling me I should be really mad and get a Surface Book

 

[kdarling]

It's clear you don't read many computer product reviews. It's not their job to find bugs, but when they encounter them it's standard practice to reach out.

According to the first reports when this happened, CR did reach out.

Apple did not respond with help until the article hit.

Apple traditionally does not often respond to outside queries. It's their M.O.

 

[69Mustang]

It's clear you don't read many computer product reviews. It's not their job to find bugs, but when they encounter them it's standard practice to reach out.

CR did reach out to Apple. It's in their article. Relevant excerpt: Apple declined to comment on our test results until they better understand the issue, but emailed this statement: “Any customer who has a question about their Mac or its operation should contact AppleCare." To be fair, Apple wouldn't want to comment without researching the issue. CR provided them with the data, Apple reviewed it and found the Safari bug.

It happens every day on things like computers and computer motherboards, and it often results in firmware modifications. The fact is that these organizations are often among the very first people to get their hands on the products outside the inner manufacturer's circle.

That's not how CR works. They don't get insider product. They buy retail. Retail for cars, computers, TV's, etc. That way they aren't beholden to the manufacturer nor do they get specially prepped products geared for testing.

 

[sudo1996]

Now back to Apple. I point out yet again that the vast majority of CR reviews for most of Apple's other stuff have been generally stellar or better. Where were you to call them garbage reviews when they were praising Apple stuff?

I called them garbage back when they recommended against buying any iPhone up until 2010. That was back when Android phones were truly ghetto. But they seem fair now.

 

[bernuli]

So Apple is saying is the short battery run time is our fault.

I believe them 100%

 

[SeaFox]

I don't think they said that at all. They said the results that CR saw shouldn't be usual based on the way they were running Safari, but they still said they found a bug that they have sense squashed.

The bug was in the software when Consumer Reports tested it. If Apple doesn't like the impact it has on testing they have no one to blame but the authors of the software, themselves, for the issue. There's no reason Consumer Reports should update testing results from what they initially found. Apple can look forward to the results of the next MacBook Pro CR reviews being more accurate.

.. Unless Apple isn't planning another new Macbook for awhile.

Also: wouldn't this mean the same bug was present in the previous MacBook Pros tested by Consumer Reports.
Apple seemed to have no issues with the test results then.

 

[HobeSoundDarryl]

I get your point, but I still don't see that it will matter. If you read their testing methodology they are really only disabling cache to simulate real world usage of going to say 25 (or hundreds) different web pages for the first time. They just disable cache because they are reloading the same page over and over again from their local test server hosting that page.

The test is to gauge how long laptop batteries last. If they turn on caches for all laptops and run the same test, that would be OK too. If they let you or me pick 25 web pages to load over and over, that would be OK too. If they raise the number to 50 web pages or 500 web pages, that's fine too as long as the test is the same for all of the laptops they test.

The point is that it's just a standard testing protocol they apply to ALL of their laptop reviews. The specifics of it are almost irrelevant as long as whatever protocols they want to use are consistently applied to the many laptops they test.

In this story, Apple interacted with them, found a software bug that contributed to the battery burn and it sounds like CR is willing to have another go at a fixed MBpro. However, some of the ongoing spin seems to revolve around the idea of how not using the cache is unnatural because using it is default... ignoring that the same standard is being applied to all of the other laptops they test too. Or more simply, great effort being made to fault CR or redirect the underlying issue away from Apple.

Use the cache or not, but the point is that if the protocol is going to change, it can't change for the just this one laptop... so that just it can perform better. It's not CR's business to help sell one corporations laptop. It's CR, not PR. And I'm not CR, nor even expert at what CR does, but I'll go ahead and concede with great confidence that that particular protocol change- if applied- will defiantly improve the battery life result for that test. CR themselves says so in the part of their quote where they explain whey they turn it off for their tests.

 

[Weaselboy]

CR did reach out to Apple. It's in their article.

Good catch. Thanks

 

[jayducharme]

Possible I am understanding this wrong, but it sounds to me like BECAUSE they used safari (and because of a bug in safari), the battery drained faster.

With over 200 posts so far, I expect someone has caught on. But just in case: CR itself apparently purposefully triggered the bug. Normal users evidently wouldn't have that problem.

Consumer Reports was using a "hidden Safari setting" which trigged an "obscure and intermittent bug reloading icons" that led to inconsistent battery life results.

 

[Weaselboy]

but I'll go ahead and concede with great confidence that that particular protocol change- if applied- will defiantly improve the battery life result for that test. CR themselves says so in the part of their quote where they explain whey they turn it off for their tests.

Of course it would, because their test reloads the same page over and over again. That is the whole reason they disable the cache so they can use their synthetic testing method.

 

[HobeSoundDarryl]

"Tougher workout" in this case is a side effect. The purpose of choosing the settings they use is to emulate real world scenario where the user hopes from one site to another. To make their tests predictable, they serve the web pages themselves. Those are limited and use the same icons/images. To make the situation closer to the real world, one has to disable caching (although, in real world, a lot will depend on the actual user's browsing style).

OK fine. Define the perfect "real world" test that will be representative of how we all use our laptops. Post it here so we can all agree that your definition is exactly how we use them. Then, after we all agree that you got it exactly right, we can all petition CR to utilize your definition as the new testing protocol for ALL laptop tests going forward. They are a very consumer-focused organization. That word is even in their name. So if we can provide them with a testing methodology that differs from their approach but that is representative of how all consumers use their laptops, I'm confident they would at least consider it.

We're out in the weeds here. The problem is NOT in the test if the same test is applied to all laptops. Had Apple made the car, would we be arguing that CR should have tested the mileage claim by allowing Apple car to roll downhill while all others get tested going uphill? Of course not. Yet here we are still trying to fault the testing (because that shines the light on CR) instead of the tested product (which very, very likely had no test bias against- or for- it... but wow how hard we are working to try to imply that).

 

[rhoydotp]

Definitely the best MacBook Pro I have owned.

Plot twist: This is his first MacBook Pro.

By the OP saying that the best he has owned, one can assume that he had a few in the past (at least one). But maybe the OP really only had this as his first.

Either way, one of you are trolling.

 

[762999]

so Safari kills battery life and Chrome kills battery life. Basically we shouldn't browse on our new Macs =/

I've heard that telnet should be ok. It's time for some starwars action: telnet towel.blinkenlights.nl

 

[Apple_Robert]

I want to be happy Apple found a bug and quickly fixed it but MacRumors is telling me I should be really mad and get a Surface Book

Ignore the biased, snide bash Apple every chance 'prepubescent type' posts. Focus on facts and your personal usage experience, and your stay here will read much better.

 

[Weaselboy]

With over 200 posts so far, I expect someone has caught on. But just in case: CR itself apparently purposefully triggered the bug.

There is no evidence CR purposely triggered the bug. They did intentionally disable caching due to the way they test using a local server, but the battery killing bug was an unintended side effect of that.

Normal users evidently wouldn't have that problem.

Correct. If you do not disable caching, you would not be impacted by this bug.

 

[69Mustang]

With over 200 posts so far, I expect someone has caught on. But just in case: CR itself apparently purposefully triggered the bug. Normal users evidently wouldn't have that problem.

Nope. You're the first and only one to suggest this. Congrats?

 

[ConnYoungy]

Doesn't explain why my machine with normal user settings enabled gets 3/4 hours battery life under casual use

 

[HobeSoundDarryl]

I called them garbage back when they recommended against buying any iPhone up until 2010. That was back when Android phones were truly ghetto. But they seem fair now.

Why are they "fair now"?

And why were they "garbage" back then?

They are Consumer Reports, not Apple Public Relations or Apple Marketing. It is not their job- and presumably not yours- to help sell Apple products. How can they be wrong when they have something negative to say but "fair" or right when they have something positive to say? It's likely the same people, doing the same job, probably running the same tests, etc.

 

[Thunderhawks]

Why are some people trying to blame Apple or Consumer reports?

I don't expect a perfect world especially for software.

CR used their standard test which they use for everybody and got varied results.
That's what they published. Nothing skewed or fishy.

Apple responded by getting the details figured out, discovering what went wrong and took care of it.
(let's hope)

End of story.

BTW: SAFARI doesn't play nice with a lot of websites (especially banks) or the other way around.
Same for programs/apps which get ported into macOS.

 

[c3pa]

Had early access to final report.

Says battery life now consistently fluctuating between 100% and 0%.

It's a pro machine after all.

 

[HobeSoundDarryl]

Of course it would, because their test reloads the same page over and over again. That is the whole reason they disable the cache so they can use their synthetic testing method.

Right. But they didn't change the test for this ONE product so they could write a negative review. The test is the test. It is applies to ALL laptops, not just this one.