LOL at the outrage in this thread. There was a software bug, they discovered it together, Apple is fixing it.
Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Consumer Reports Retesting MacBook Pro Battery Life After Apple Says Safari Bug to Blame
- Thread starter MacRumors
- Start date
- Sort by reaction score
See headline. Desperately think of something stupid to post.
Go Go Go! Do not bother reading article.
Go Go Go! Do not bother reading article.
this is probably why some people on beta 3 have seen increase in battery life
I would imagine they would show disparity if they were re-tested with the current Safari (in Sierra), although possibly of less magnitude due to older power management technology.
They did do additional testing with Chrome and got consistent battery life, which confirms it was a software bug and not a hardware lemon. Are you expecting CR to look thru source code they don't have? Do some software profiling? Run a debugger?
You're totally right. Consumers would never drive their car straight into a wall, so why do people crash test cars. Horrible methodology.
HobeSoundDarryl
macrumors G5
OK. But CR themselves admits that disabling caches is done to give batteries a tougher workout. Here's their quote:
We also turn off the local caching of web pages. In our tests, we want the computer to load each web page as if it were new content from the internet, rather than resurrecting the data from its local drive. This allows us to collect consistent results across the testing of many laptops, and it also puts batteries through a tougher workout.
Fixing the icon bug but leaving the existing protocol in place does as you suggest. However, using the cache as many here are suggesting as default would demand using the same cache on everyone else's laptops too. Since their batteries would not be subjected to "a tougher workout" (too), their battery life seems likely to (also) improve.
CR _did_ try to reach out to Apple when they first got the wacky results. This was mentioned in all the original articles on this topic.
Apparently Apple did not jump at the chance to help, until the review actually came out.
As for the testing method, they wanted the cache turned off, so that the machine is actually going out to fetch the page each time.
It's not the only way to do it, but the fact that Apple has never seen this bug, raises the question of how they do their own web time testing. Do they let the cache kick in every time? Or are they using a rolling parameter or server side header settings to get around it?
Last edited:
I think it's hilarious how they keep stressing this pompous statement in every media blurb surrounding these devices.
I remember the days when Apple simply released a decent product and let people form their own opinions.
As for CR, I hope they make a new web test using their own server that generates random webpages that aren't in the cache on every hit. Then we'll see if it really was a bug.
The alternative is creating hundreds of thousands of distinct webpages on their internal server: the entire point of the test is active web browsing, not cache retrieval.
I don't think we yet know if this was a real "bug", or just that Safari is awful at wifi battery consumption.
Real headline:
Apple confirms Macbook Pro is not for professionals, says using Safari Developer mode is not realistic on a Macbook Pro.
Thanks. Now give us a real Pro laptop.
Apple confirms Macbook Pro is not for professionals, says using Safari Developer mode is not realistic on a Macbook Pro.
Thanks. Now give us a real Pro laptop.
It's clear you don't read many computer product reviews. It's not their job to find bugs, but when they encounter them it's standard practice to reach out. It happens every day on things like computers and computer motherboards, and it often results in firmware modifications. The fact is that these organizations are often among the very first people to get their hands on the products outside the inner manufacturer's circle. You can ignore that if you wish, but it's very, very common practice. Apparently CR is too inept to do this.
And don't take this as an Apple shill so much as a CR bash. I have felt they were garbage across the board for many years. I don't own a MBP and it's unlikely that I ever will own another.
I get your point, but I still don't see that it will matter. If you read their testing methodology they are really only disabling cache to simulate real world usage of going to say 25 (or hundreds) different web pages for the first time. They just disable cache because they are reloading the same page over and over again from their local test server hosting that page.
"Are you expecting CR to look thru source code they don't have? Do some software profiling? Run a debugger?"
Or even wonder why CR's tests which are properly being executed (and presumably monitored, observed, and supervised) were able to pull fresh web pages off their server, in some trials, for 12, 14, 16, 18 1/2, and 19 1/2 hours straight?
Well, just a few thoughts from a web designer/developer...
It seems reasonable to disable caches provided they disable caches on all tested machines and confirm that the setting is being honored. With caches enabled many reloads would reference cached data on the computer which would fall materially short of real-world web browsing. That said, in real-world web browsing caching does save considerable amounts of data (e.g. browsing to individual pages on a website where shared scripts and style sheets are not loaded from cache, or even across browsers where a script like jQuery may be stored on a shared external resource like Google APIs). So incongruity either way—it would have to be made up for, to some extent, in how frequently the test reloads web pages and what pages are used.
Disabling caches shouldn't independently produce the swing in battery life Consumer Reports experienced in their lab. That would, one assumes, be a product of the 'icon' bug Apple has referenced.
Interesting results with Chrome. I understand why Consumer Reports shifted over after seeing odd results in Safari. But Chrome typically is far less energy (and memory) efficient than Safari on macOS.
I believe Consumer Reports handled this very irresponsibly. When they produced these wildly inconsistent results, and could not explain those results, they should have contacted Apple and taken steps to sort out the problem. And even if publishing deadlines prohibited them from fully resolving the problem (even this being questionable without eliminating the possibility that the problem ties to their procedure) they should have at least communicated with Apple and drawn no material conclusion based on the data without first understanding its relevance to consumers.
It seems reasonable to disable caches provided they disable caches on all tested machines and confirm that the setting is being honored. With caches enabled many reloads would reference cached data on the computer which would fall materially short of real-world web browsing. That said, in real-world web browsing caching does save considerable amounts of data (e.g. browsing to individual pages on a website where shared scripts and style sheets are not loaded from cache, or even across browsers where a script like jQuery may be stored on a shared external resource like Google APIs). So incongruity either way—it would have to be made up for, to some extent, in how frequently the test reloads web pages and what pages are used.
Disabling caches shouldn't independently produce the swing in battery life Consumer Reports experienced in their lab. That would, one assumes, be a product of the 'icon' bug Apple has referenced.
Interesting results with Chrome. I understand why Consumer Reports shifted over after seeing odd results in Safari. But Chrome typically is far less energy (and memory) efficient than Safari on macOS.
I believe Consumer Reports handled this very irresponsibly. When they produced these wildly inconsistent results, and could not explain those results, they should have contacted Apple and taken steps to sort out the problem. And even if publishing deadlines prohibited them from fully resolving the problem (even this being questionable without eliminating the possibility that the problem ties to their procedure) they should have at least communicated with Apple and drawn no material conclusion based on the data without first understanding its relevance to consumers.
''
Have you actually used a Microsoft Surface Book or Surface Pro 4 next to one of these MacBook Pro laptops? If not, do yourself a favor and go into Best Buy and try them out. The build quality on the Macs is light years ahead of Microsoft. There's really no comparison.
Actually, the setting is not exposed to normal users, so there is controversy on whether it should have been used as part of a test meant to simulate user behavior.
They had anomalous results on every laptop shipping with this OS version/browser version. It makes sense to suspect an issue with the testing practice (or the testing practice exposing an issue with the environment)
I understand CR does not want to be influenced, but I agree one would think if you see test results that are very inconsistent and so at odds with the manufacturer's claim, you would want to ask the company if they have some idea why this is happening. Just in the interest of providing all the information to the readers if nothing else.
HobeSoundDarryl
macrumors G5
And I've felt just the opposite. In fact, I feel CR is about the LAST, very objective source of Consumer Product reviews anywhere.
What you apparently seek from them is probably impossible. They review tons of products. Holding them to figuring out "bugs" in all products that seem like they should get a better review would mean they probably couldn't get any kind of reviews out.
Take Apple out of it. When I'm buying a new Microwave or Tires or a Printer, I'd much rather see their review than- say- reviews by very biased people on a site like this... or what seems extraordinarily likely to be some purchased Amazon "objective" reviews... or reviews on sites that are in the business of selling ads and thus does want eyeballs to maximize revenue (and does have advertising buyers to try to keep happy).
It's fine for you to feel they are "garbage". I don't. In my experience, their reviews have been excellent at helping me buy all kinds of stuff that have generally performed very well. Glad I didn't buy the stuff that they rated very poorly. And glad they exist so I had some way to know many such (bad reviews) products from the few they rate so highly.
Now back to Apple. I point out yet again that the vast majority of CR reviews for most of Apple's other stuff have been generally stellar or better. Where were you to call them garbage reviews when they were praising Apple stuff?
That was my first thought, and I'm surprised that you're the first person in this thread who has mentioned it. What percentage of people experiencing poor battery life are using Safari with caching disabled? If those people enable caching, does battery life dramatically improve? What about everyone else? Have people with battery-life issues experimented with their settings, which software they're using, under what conditions, etc.? Yes, I know people (rightly) expect an expensive new product to "just work," and that troubleshooting takes time and effort, but until people start narrowing down the circumstances under which they're experiencing poor battery life, all we have are comments of "I'm getting terrible battery life!" and "Well, my battery life is great!" I've been using Apple products since 1991, and, historically, their laptop computers have gotten great battery life. There may be exceptions that I'm forgetting, but something seems to be going on with this particular model. Is it a software issue that would act the same on all new MBPs with identical settings? Is it a hardware issue that affects only some Macs? Has anyone experiencing battery-life issues exchanged their MBP for another one? If so, do they experience the same problem, or do they now get the rated battery life?
"Tougher workout" in this case is a side effect. The purpose of choosing the settings they use is to emulate real world scenario where the user hopes from one site to another. To make their tests predictable, they serve the web pages themselves. Those are limited and use the same icons/images. To make the situation closer to the real world, one has to disable caching (although, in real world, a lot will depend on the actual user's browsing style).
That's my point. If CR is testing a dishwasher that just flat out doesn't work, are they going to go to the manufacturer and say hey.. I think maybe there's something wrong with this unit... or do they just publish a report saying don't buy this one, it doesn't work.
