You know, one thing you heard over and over in these forums is "Benchmarks Don't Matter." Well that was until the iPhone started posting better benchmarks. Then all of a sudden benchmark scores were being posted all over the place. And all of a sudden Benchmarks had significance. So now with the GS4 sure to shatter a few benchmarks will these forums go back to chanting the "Benchmarks Don't Matter" mantra?
Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
So will Benchmarks still matter?
- Thread starter Timzer
- Start date
- Sort by reaction score
I've always believed benchmarks don't matter. Benchmarks don't really measure the true experience of using your smartphone. I've been using Android for some time now, and I can definitely say, when it comes to Android Smartphones, it really matters which one you buy. Even amongst the high end Android phones, they're not all the same performance wise even though they have similar specs.
But to answer your question, I'm confident to say for those that were going on about Benchmarks of the iPhone, will suddenly become absent.
But to answer your question, I'm confident to say for those that were going on about Benchmarks of the iPhone, will suddenly become absent.
With technology having progressed as far as it has, I think we are at the point that no matter what, a high-end phone will have "fast" performance. The point we are at now, benchmarks are merely bragging rights for people that are into that sort of thing.
Yes & no.
When looking at a new flagship, if for example HTC put out the 'one' and it benchmarked 11,000 on AnTuTu (same as SGS3) people would describe it as poor regardless of whether it felt snappy or not, especially if the SGS4 scored around 21,000 on the same Benchmark.
It wouldn't matter if all the reviews of both devices said they felt the same in daily usage, that discrepancy in a benchmark would likely act as a crucial swing in the SGS4's favour, more so than manufacturers software.
They matter for psychological purposes in consumers (perception) and marketing. (2 x as powerful claims etc...)
-
When looking at a new flagship, if for example HTC put out the 'one' and it benchmarked 11,000 on AnTuTu (same as SGS3) people would describe it as poor regardless of whether it felt snappy or not, especially if the SGS4 scored around 21,000 on the same Benchmark.
It wouldn't matter if all the reviews of both devices said they felt the same in daily usage, that discrepancy in a benchmark would likely act as a crucial swing in the SGS4's favour, more so than manufacturers software.
They matter for psychological purposes in consumers (perception) and marketing. (2 x as powerful claims etc...)
-
Ive never really cared about benchmarks. There are so many diff scores i dont know what they are for anyways and ive always said that.
Thing is when the GS3 came out, it was being compared to the 4S and iPhone people said it wasnt fair and to wait til the iP5 came out. Ok, it is out and now some Android people say it isnt fair to compare the iP5 to the GS3, wait til GS4 comes out,....LOL.
Really? Yes it is fair and we are never gong to have them out at the same time so you compare what is out so you compare what is the flagship at the time.
Whats my point? Well im just saying who cares? I like a bigger screen and more functions and customizable things. I get that from Android and i dont really care which one is 1 or 2 seconds faster than the other. When Apple can do the things above, they will be considered and only when they come to T-Mobile which is supposed to be this year.
I dont mind Polycarbonate plastic or if it is aluminum. Gets a case either way but i do care to have a MicroSD card which will come with a removable back.
Benchmarks don't matter in the mobile world, unless you're comparing things from the same line AND you actually get a good use of your mobile device, like in, "advanced" use.
For instance if you take the 4s vs the 5, then you can see there is a "speed performance" boost on the 5 and if you use any app that you feel kinda "slow" on the 4s chances will be user experience will improve on the 5.
I know they post things like "Oh it now opens pages 0.5 seconds faster, and if you open X pages a day then WOAAA you're shaving off 10 minutes per day" and placebo comments like that.
When you are not comparing things from the same family... then trust me, it's just "bragging rights" like a former user said.
Of course the mobile benchmarks nowadays are the new black... or maybe the new megapixel (back in the time -maybe still relevant nowadays?- when people got cameras just because of their megapixel specs, when you can have a pro 5 MP camera that will put your cheap 12 MP one to shame).
At work, people are stupid enough (and I say enough, because we're mostly computer scientists there... imagine "regular" people) to buy whatever comes out next just because it has a "faster processor" and stuff like that. Then all they use is their twitter and facebook app.
To put things in perspective, the poster above me talks about stuff that makes him buy Android devices.
From my perspective, I use Auria, Audiobus, Sunrizer (and the list goes on), to name example and latency on Android is still a joke, so from a "mobile musician" perspective I couldn't care less about any Android device, even if it was the most powerful one in specs out there, just because it's a worthless system for music production on the go, from my perspective.
So see? It's all about what you really like and need, not about some nerd scores.
It's all placebo... expensive one
For instance if you take the 4s vs the 5, then you can see there is a "speed performance" boost on the 5 and if you use any app that you feel kinda "slow" on the 4s chances will be user experience will improve on the 5.
I know they post things like "Oh it now opens pages 0.5 seconds faster, and if you open X pages a day then WOAAA you're shaving off 10 minutes per day" and placebo comments like that.
When you are not comparing things from the same family... then trust me, it's just "bragging rights" like a former user said.
Of course the mobile benchmarks nowadays are the new black... or maybe the new megapixel (back in the time -maybe still relevant nowadays?- when people got cameras just because of their megapixel specs, when you can have a pro 5 MP camera that will put your cheap 12 MP one to shame).
At work, people are stupid enough (and I say enough, because we're mostly computer scientists there... imagine "regular" people) to buy whatever comes out next just because it has a "faster processor" and stuff like that. Then all they use is their twitter and facebook app.
To put things in perspective, the poster above me talks about stuff that makes him buy Android devices.
From my perspective, I use Auria, Audiobus, Sunrizer (and the list goes on), to name example and latency on Android is still a joke, so from a "mobile musician" perspective I couldn't care less about any Android device, even if it was the most powerful one in specs out there, just because it's a worthless system for music production on the go, from my perspective.
So see? It's all about what you really like and need, not about some nerd scores.
It's all placebo... expensive one
They don't matter a ton to me. They do have some importance but I gauge real world performance. Much more practical. Anything like what's in the nexus 4, iPhone 5, htc one, nexus 10, iPad 4 etc. is more than fast enough for today's apps and will be for quite some time until apps catch up. Even the A5 is still holding its own. Thanks to the advancements already made In the desktop and laptop arena mobile is able to advance much faster.
I would say that they matter, but they must be looked at in context, and you have to keep an open mind and remember that they do not always tell the whole story.
For instance, while benchmarks may show that X phone loads a browser page much faster than Y phone, just how meaningful is that in real world terms? Eg: how many seconds does that shave off?
For instance, while benchmarks may show that X phone loads a browser page much faster than Y phone, just how meaningful is that in real world terms? Eg: how many seconds does that shave off?
I dont understand what most of them mean anyway.
I think we have reached the level of sophistication with regards to smart phones for the real life user differences in performance to be largely unnoticeable.
I think we have reached the level of sophistication with regards to smart phones for the real life user differences in performance to be largely unnoticeable.
They matter but shouldn't. It take a huge jump in benchmarks to notice a difference in how the phone run yet people go on about minute differences in benchmarks.
It's mostly for bragging rights. That's like people that over clock their phones. It's not like apps are opening too slow or something, a good majority of them just want to see how fast their phone can be in benchmarks. Bragging rights.
If benchmarks really mattered then the ability to over clock and shatter current benchmarks shouldn't be considered "cheating".
It's mostly for bragging rights. That's like people that over clock their phones. It's not like apps are opening too slow or something, a good majority of them just want to see how fast their phone can be in benchmarks. Bragging rights.
If benchmarks really mattered then the ability to over clock and shatter current benchmarks shouldn't be considered "cheating".
Benchmarks don't matter in the mobile world, unless you're comparing things from the same line AND you actually get a good use of your mobile device, like in, "advanced" use.
For instance if you take the 4s vs the 5, then you can see there is a "speed performance" boost on the 5 and if you use any app that you feel kinda "slow" on the 4s chances will be user experience will improve on the 5.
I know they post things like "Oh it now opens pages 0.5 seconds faster, and if you open X pages a day then WOAAA you're shaving off 10 minutes per day" and placebo comments like that.
When you are not comparing things from the same family... then trust me, it's just "bragging rights" like a former user said.
Of course the mobile benchmarks nowadays are the new black... or maybe the new megapixel (back in the time -maybe still relevant nowadays?- when people got cameras just because of their megapixel specs, when you can have a pro 5 MP camera that will put your cheap 12 MP one to shame).
At work, people are stupid enough (and I say enough, because we're mostly computer scientists there... imagine "regular" people) to buy whatever comes out next just because it has a "faster processor" and stuff like that. Then all they use is their twitter and facebook app.
To put things in perspective, the poster above me talks about stuff that makes him buy Android devices.
From my perspective, I use Auria, Audiobus, Sunrizer (and the list goes on), to name example and latency on Android is still a joke, so from a "mobile musician" perspective I couldn't care less about any Android device, even if it was the most powerful one in specs out there, just because it's a worthless system for music production on the go, from my perspective.
So see? It's all about what you really like and need, not about some nerd scores.
It's all placebo... expensive one
I think you pretty much nailed it with comparing different platforms. Two phones running different os's is hard to compare benchmarks. Really anything from mid 2012 on is going to feel great regardless of what a benchmark says.
Also, that the things that are important now are the actual cameras or feature that makes us more productive... Not cpu and ram since they are already up to par.
Benchmarks don't matter, never have, at least to me. Not when it comes to comparing devices.
As long as the device runs smoothly and is fast, that's good enough for me. I don't care if it's beat in a benchmark by another device that actually runs less smoothly (which is often the case).
As long as the device runs smoothly and is fast, that's good enough for me. I don't care if it's beat in a benchmark by another device that actually runs less smoothly (which is often the case).