Samsung Again Caught Inflating Benchmarking Scores, Phil Schiller Calls 'Shenanigans'

[i.mac]

yeah, well I can actually buy one of those.
Can't say the same for the 5s.

Thank you. More 5s for us. Been waiting here for a while now...

 

[crazy dave]

So if they developed code to run this around other apps (like the article vaguely mentions it does) it would be OK?

You're trying to tell me someone on XDA isn't going to be able to write up some code to inject into the system that would give the app they develop a boost if they wanted?

CPU governors have been available on Android since pretty much Day 1. I have been able to *artificially* boost the CPU power for a task based per app or overall. Yes, battery life suffered. Yes, performance increased. But at its core, which is the entire point here, the CPU was capable of doing it - just like this benchmark proved. The CPU did the work it was asked to do. The app was NOT compromised to get around tests, the CPU did the work.

The reason this is of dubious morality is because they are using this code to make it look like their CPU is faster than their competitors ... even those who use the exact same CPU (LG) but didn't put in a governor to inflate the CPU speed. Some reviewers are even glowing about the benchmark scores and noting how much faster it is than the LG despite using the same CPU - when in fact it is no faster at all, it is simply running an execution path optimized for benchmarking. Any other phone could do the same and get higher scores. So this isn't about trying to show what the CPU is truly capable of if your app is optimized or runs a governor, this is about trying to inflate their scores over their competitors (mostly their Android competitors like LG) to make their phone look faster compared to them than it really is for marketing purposes.

 

[Iconoclysm]

Don't forget that most Apple devices are crammed full of Samsung stuff (including the 5s)...

Less than 15% of the components of any Apple device are made by Samsung.

 

[Northgrove]

Haha, this shows how much of a different game Samsung is actually playing. Imagine if Apple did this -- they'd be ripped to shreds due to the expectations. With Samsung? "Ah, I see they did it again."

 

[SPUY767]

This is pretty hilarious actually. Why not just always run the CPU this way

Because the phone would overheat and your battery would last about an hour?

 

[TMay]

yes, at twice the clock rate. Its a tradeoff requiring more power/heat dissipation. Really a shocking testament to how good the A7 w/ARMv8 is (assuming that's the 64-bit score you are looking at and not the 32)

I don't think that Geekbench supports ARMv8 so it must be 32 bit. Look at the release date for the latest Geekbench version vs when Apple released support for ARMv8 in XCODE to developers.

 

[Michael Goff]

Haha, this shows how much of a different game Samsung is actually playing. Imagine if Apple did this -- they'd be ripped to shreds due to the expectations. With Samsung? "Ah, I see they did it again."

No, if Apple did it then we wouldn't be talking about it at all. News sources would brush it under the rug.

 

[639051]

I can see why Samsung has to run the CPU cores faster, have you ever played with one of their devices running TouchWiz? Sure, it is fast at first .. but just wait .. the lag comes and never wants to leave.

I just posted my GS4 for sale, while I like Android I have found that the iPhone just keeps me coming back.

 

[g88bo]

The reason this is of dubious morality is because they are using this code to make it look like their CPU is faster than their competitors ... even those who use the exact same CPU (LG) but didn't put in a governor to inflate the CPU speed. Some reviewers are even glowing about the benchmark scores and noting how much faster it is than the LG despite using the same CPU - when in fact it is no faster at all, it is simply running an execution path optimized for benchmarking. Any other phone could do the same and get higher scores. So this isn't about trying to show what the CPU is truly capable of if your app is optimized or runs a governor, this is about trying to inflate their scores over their competitors (mostly their Android competitors like LG) to make their phone look faster compared to them than it really is for marketing purposes.

Instead of innovating, they're copying and trying to cheat the benchmark. Meanwhile, Apple creates an M7, A7 and a whole lot of new things which are thought through and beyond known limits...

 

[FrozenDarkness]

lol, samsung lying and its nutjob fans defending on macrumors

 

[Michael Goff]

Instead of innovating, they're copying and trying to cheat the benchmark. Meanwhile, Apple creates an M7, A7 and a whole lot of new things which are thought through and beyond known limits...

64-bit is not innovation. -_-

 

[skorpien]

For people saying "What's the big deal?" here it is:

1. Those processors aren't meant to run at that spec for prolonged periods of time due to battery drain and heat issues.

2. Samsung are temporarily disabling these restrictions so that their products perform better at certain benchmarks, therefore making their products seem like they're much better than the competition.

3. Yes, you CAN make your Samsung perform at those levels, but that would require rooting it and tweaking the clock speed (thus decreasing your battery life and making the device run hotter).

The MAIN point is number 2. It makes their products appear superior to products that are identical in nearly every aspect, the only difference is that the competitors don't have code in their version of Android that lets them "cheat" at benchmarks.

They're not the innocent little company that you are making them out to be.

*facepalm*

 

[Jman13]

I don't think that Geekbench supports ARMv8 so it must be 32 bit. Look at the release date for the latest Geekbench version vs when Apple released support for ARMv8 in XCODE to developers.

The current Geekbench 3 supports the A7 in 64-bit mode with all the ARMv8 instructions. The latest release was 9/17/13. The description in the app store says, "Geekbench 3 is now optimized for iOS 7 and includes full support for 64-bit processors."

 

[Macboy Pro]

what on earth are you talking about?

cheating is cheating. only cheaters think cheating is ok.

The point is that Apple cheats on their claims also. IE. Show me a macbook pro with 7 hours of real battery life. PLEASE. I can't find one with 5 hours. But they claim 7. how is that different from Samsung tweaking their product to get the best benchmark in order to claim it.

Good grief I love Apple, but I can not stand the following of people who think Apple is the royal family of businesses. They are just a crooked as the rest and it is hypocritical to make the claim like this.

 

[nick_elt]

Read every post response to me when I made a similar point.

It's impossible to have an unbiased opinion when it comes to Samsung/Apple. You're one side or the other on this forum.

i have both, i dont see why ppl get all biased, makes no sense to me. Not even Apple is biased where they get their components from. Its just business to them.

 

[pk7]

FWIW the iPhone 5s outscores almost all of the non-boosted Note 3 scores, with the exception of floating-point (in which it scores only 17 points lower).

Samsung are insecure little cheapos. Proud to say that I've never owned a Samsung smartphone and that I never will.

 

[newagemac]

So if they developed code to run this around other apps (like the article vaguely mentions it does) it would be OK?

You're trying to tell me someone on XDA isn't going to be able to write up some code to inject into the system that would give the app they develop a boost if they wanted?

CPU governors have been available on Android since pretty much Day 1. I have been able to *artificially* boost the CPU power for a task based per app or overall. Yes, battery life suffered. Yes, performance increased. But at its core, which is the entire point here, the CPU was capable of doing it - just like this benchmark proved. The CPU did the work it was asked to do. The app was NOT compromised to get around tests, the CPU did the work.

Well that's perfectly fine as long as the Note's battery life and unbearable heat is reported the same way. So report the battery life of the Note as 30 minutes or whatever under intensive tasks while the iPhone 5S and LG G2 get 8 hours or so. And also report that unlike other phones when playing game so and so or other intensive tasks, the Note is unbearable to even hold in your hand.

How about that? What Samsung is trying to do here is have their cake and eat it too. And nobody is buying it.

 

[bbeagle]

No, if Apple did it then we wouldn't be talking about it at all. News sources would brush it under the rug.

News sources never mentioned antennagate? foxconn? tax dodging? tracking your location? e-book price fixing? None of those were ever on the news? oh, ok.

 

[GRouND ZeRo]

Sorry Johnny...

Samsung is a joke lol. Apple will always be the superior company. If only there was a way to make others see this...like to baptize them or something.

I find Apple is a Joke when it comes down to "OUTDATING" perfectly working hardware.... I own a iPhone 3G and can't get any decent apps for it. I used to own a Dual Core Intel MacBook and tried to edit 1 hour worth of video(Firewire from Cannon DV Camera) and it took me over 5 HOURS to make a DVD! I literally gave the MacBook away!(It takes me less than an hour to import and burn a DVD with my PC!) I don't think others will see this other than a hole in their pocket!

 

[lilo777]

Your view is plain incorrect.

The reason to run benchmarks is to measure what performance the device will produce in normal use. Samsung took steps to show 20% higher values without telling anyone. That's plain old cheating.

If Samsung had gone to Ars Technica and told them "please test how fast this phone can run in a special benchmarking mode", that would have been absolutely fine.

I happen to drive a car with a Diesel engine. The engine is quite deliberately limited. Over 2500 rpm, torque and horse power both drop. The engine itself could deliver the same torque at higher rpm, but that would very quickly damage the engine. The same car could run significantly faster if you were willing to accept that the engine won't last 200,000 miles but only 20,000 or only 200. If the manufacturer wrote "we removed all limits from the engine and achieved 150mph and managed to stop before the engine exploded" that's fine. If they wrote "the top speed is 150 mph" without mentioning engine damage, making their customers believe that they can also achieve that speed, that would be cheating.

You are plain wrong. It's not a benchmarking mode. here are other apps (camera app, for example) that the phone runs in this mode. On the MBPs with turbo mode, applications that task CPU for a significant period of time will also trigger disabling of turbo mode and they will not be run the same way the short running benchmarks are run. Is Apple cheating too?

The only legitimate issue here is that right now only Samsung can specify which apps may run in this "turbo" mode. But it sort of clear why they would not allow user to do it. Perhaps they might work with major app vendors to allow this mode for selected apps.

 

[xensaenigma]

Yes he's quite good at whoa is me, we're the victims.

It's just business, man up and quit whining Apple. If you're still sad look at the billions you have. With Apple greed knows no limits.

What would you say if Apple was found to have faked its numbers?

 

[WestonHarvey1]

Defending Samsung on this really is silly.

ATI did the same thing with their drivers once. They specifically manipulated Quake 3 benchmarks by turning down the graphics quality when the process was named "quake3.exe". By simply renaming the executable to something else, you could get the higher graphics quality you expected.

That's exactly what Samsung did here.

 

[apolloa]

Apple executive Phil Schiller -- senior vice president of Worldwide Marketing and the most prolific tweeter amongst Apple's senior staff -- linked to the Ars article in a tweet, saying only "shenanigans".

Soooooo..... the executive Of Apple publically slanders it's biggest supplier of components, wow, School boy tit for tat is the Apple way these day's then? They are fast becoming even more hypocritical and truly unprofessional and immature than ever now.

I do hope one day samsung would just tell Apple to F Off! Or perhaps Apple feels it MUST slander the competition now seeing as iOS7 is an utter disaster and making people literally sick! Can't remember the last Galaxy smart phone that made people physically sick?

Oh and yeah I work for Samsung and get paid tons of moolah for posting stuff on here

 

[kdarling]

Many companies have done this and stuff like this in the past, and everyone of them have gotten their allotted share of critique.

Companies like Intel famously tweaks their compilers to produce better code just for some benchmarks, and ATI and nVidia have had special drivers so that certain benchmark runs in certain games ran optimally.

Yep, compilers and video cards are infamous for checking for benchmarks. Ditto for browsers.

Then there's the opposite problem. Sometimes companies come up with their own benchmark that favors their own products. Microsoft has done that.

Some even critiqued Apple when they compared G5's to Pentium 4s.

Ah yes, the 2003 Power Mac G5 benchmark "scandal". Where Steve Jobs quoted tests from a company that they had hired to "prove" that the G5 outperformed Intel desktops. It turned out they used Apple supplied tools to tweak the performance, and also changed the memory malloc code to work faster.

It's silly for any company to try to do this kind of manipulation these days, because it's going to get caught. Samsung in particular should be trying to improve its image, instead of allowing its programmers to code for benchmarks.

 

[olowott]

come on they copy almost everything

what will stop them from hmmmm tweak the score board just aLOT.