Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

Nearly All Mobile Device Makers Cheat on Benchmarks, Except Apple and Motorola

Like the imaginary sales vs shipping comparisons, I'm in favour of cheating on benchmark tests. It exposes all the players for who they really are…

Faking those tests is a great distraction for Samcopy, you know, instead of making good phones.

And the bogus results are greedy money for the media/bloggers talking-up Apple's competitors, because they spend 10x what Apple does in advertising.

That's the only reason Samcopy, et al. get favourable news. It's not because of the quality or innovation in their products, it's by how much they spend in advertising. Nothing more complicated than that.

Time to stop believing they're in the news business. They're in the advertising business. They'll write whatever it takes to get you to see their ads. Hence the state of the media/blogosphere.

It's extortion to get Apple to play the media's game & spend more on advertising.

Like Wall St - if Apple wants to be in their club, they have to give up Job's principles and pay dividends, make cheap phones and whatever Wall St makes profits from. Again, nothing to do with running a good company or making good products. Extortion, plain and simple.

----

NoAgenda - learn something!
 
Dark Dragoon said:
ArsTechnica looked into that on the Note3, there is a file with a list of benchmark applications to trigger the behaviour based on the name of the application. If you rename your benchmark application then it doesn't boost it.
Click to expand...

That's more like it. ;) Anand skimped over it. :mad:

That is also why in PC gaming, the users have stopped relying on pure benchmarks - people started relying on real game performance. That makes it fair and useful: if CPU/GPU manuf cheats in those games, then I do not care why, I got more fps in my games!

(There is another reason why benchmarks are as good as dead on the PCs: the manufacturer bias. Some run better with Intel, some with AMD/ATI, some with nVidia.)

Probably the mobile game developers should start adding the so-called "demo" mode to their games which would allow to see how good game actually performs on the particular device. (Plus few quantifying numbers in the end: min/max/avg fps, cpu/sys/wallclock time.)
 
Bottom line Samsung is dirty!!

If you're going to horse-trade on patents, it might help to know how much other companies have paid for them, no? Samsung has now been accused of corporate spying for exactly that purpose. The company's legal negotiator, Dr. Seungho Ahn, apparently told Nokia that its terms with Apple "were known to him," despite the fact they were marked "highly confidential -- attorneys' eyes only." This means they should have been for Samsung's outside counsel only, and strictly off-limits for gaining leverage in any negotiations. Court documents show that up to 50 Samsung employees were given non-redacted copies of Apple agreements with not only Nokia, but Ericsson, Sharp and Philips, too. While that's plenty serious, US judge Paul S. Grewal ordered Samsung to release further selected documents and emails to Apple and provide testimony from Dr. Ahn. The result of that will determine the severity of any sanctions, and perhaps make Dr. Ahn regret another thing he told Nokia: "all information leaks."
 
alhedges said:
Umm, no. Apple designed the chip, as is well documented and understood by anyone involved in this area. You are simply lying.
Click to expand...

You simply have no understanding how the manufacturing works. You mixing up Samsung Semi and Samsung Electronics also suggests poor knowledge of the companies: unlike Apple, Samsung isn't vertically integrated company; there are many, loosely cooperating Samsungs.

Otherwise. In semiconductor manufacturing it is impossible to separate design and manufacturing. (Unless old and established manufacturing process is used, what is not the case here.) To whatever extent Apple has designed the chip, at some (very early) point of time either Samsung Semi's engineers came to Apple, or (more typically) Apple's engineers came to Samsung Semi, and made the redesign of the design to exploit the new manufacturing process, to test several first generation of the chip, to make a specsheet of chip to allow other Apple's engineers to integrate the chip into the final product.
 
philips said:
You simply have no understanding how the manufacturing works. You mixing up Samsung Semi and Samsung Electronics also suggests poor knowledge of the companies: unlike Apple, Samsung isn't vertically integrated company; there are many, loosely cooperating Samsungs.

Otherwise. In semiconductor manufacturing it is impossible to separate design and manufacturing. (Unless old and established manufacturing process is used, what is not the case here.) To whatever extent Apple has designed the chip, at some (very early) point of time either Samsung Semi's engineers came to Apple, or (more typically) Apple's engineers came to Samsung Semi, and made the redesign of the design to exploit the new manufacturing process, to test several first generation of the chip, to make a specsheet of chip to allow other Apple's engineers to integrate the chip into the final product.
Click to expand...

Look, Apple invested $300 million to buy PA-Semi. Do you think they did that just for fun?

99% of the work on A7 is Apple's design. One percent is adapting the design to the limitations of Samsung's process.

And all that rhetoric doesn't change one bit about the fact that Samsung was caught, for the second time in a year, cheating on benchmarks.

----------
Delegator said:
Do people really care about benchmarks for phones?
Click to expand...

I don't because I quite expected some manufacturers to cheat. But many people did care. But let's say there is only five percent of Android customers who want to decide between an HTC phone and a Samsung phone, decided that they have close to equal value, and therefore take the benchmark results to decide what to buy. That's massive number of people buying Samsung instead of HTC because Samsung was cheating.
 
Speaking of benchmark cheating, how about the '64 bit processor and twice the power' when you only have 1 gig ram, and most programs won't even be 64 bit in a long time?
 
Typical

This is typical of Samsung & other companies to try and compete with Apple.
Apple makes such quality products it's hard to show your product better then Apple products. I've always wondered when I checked out friends S3s & S4s why they seemed slower and most of all they lagged a lot. Needless to say their plastic crap and feel cheap in the hand! All but one friend that had switched from an iPhone to S4 have switched back and only did it because they wanted a Samsung's bigger screen. Turned out it wasn't worth it, but I will admit the one that has not switched back to iPhone is happy with his S4. I'll give it 6 months from now until he's fed up with Samsung & Android.:cool::apple:

I m sure the Samsung phones are ok but obviously not what Samsung claims.:cool:

:apple::apple::apple:
 
Oletros said:
Samsung is not doing the benchmark and using that benchmark to do any marketing so unfair competition doesn't has nothing to do

And, actually, the phone is as fast as they say, they are not overclocking it, they are using the full CPU speed.
Click to expand...

Wrong on two accounts: Marketing isn't just advertising. Marketing is all possible ways to get the message about your product out to the public. Therefore, cheating on benchmarks _is_ marketing. And in Germany, "unfair competition" is a very broad term. Misinforming customers is just one form of unfair competition, and whether it happens in advertisements where it would be quite obvious or in a more underhanded form like Samsung's benchmark cheating doesn't really matter.

Second, they are using more than the CPU speed that is available to the customer in normal operation. They are using a CPU speed that is specifically reserved for applications named "GeekBench 3" or one of a dozen other names. If the phone cannot handle using what you call "full CPU speed" continuously, then using it in a benchmark is cheating.
 
Applesexual2 said:
"For 32/28nm, Samsung and its partners were the first to introduce the Foundry HKMG strategy. At 32/28nm, a Gate-First HKMG was selected since the scheme has proven to meet market and customer needs achieving superior area scaling and preservation of layout styles without complex restricted design rules."

Do the words "restricted design rules" not mean anything to you?
Who should I trust, some blind Apple fanboy or Samsung, the leaders in manufacturing tech?
Click to expand...

Let me just say that I have worked at a company creating graphics and combined graphics/processor chips for many years and learned a bit how this business works. So I can say with a very clear conscience that you have absolutely not a clue of what you are talking about, and calling people "blind Apple fanboy" is rich coming from you.

Second, Samsung is not the leader in manufacturing tech. That's Intel, IBM, TSMC. And unfortunately, besides being (not quite) the leader in manufacturing tech, Samsung is also the current leader in benchmark cheating. Which doesn't change one bit that you are looking at their advertisements, believe them without questioning, and most importantly completely misinterpret what they are saying.

In Formula 1, there is a massive set of rules set by the FIA that the car manufacturers and drivers have to follow. According to your interpretation, the FIA and not the manufacturing teams or the drivers are responsible for the speed of the cars. So we really should just give all the points in the races and the world championship to the FIA. Just to make a bit more clearer to you (everyone else knows already) that what you are saying is nonsense.
 
Last edited:
iSee said:
Sweet, kicks! Where can I get me a pair of those Adadis?
Click to expand...

A few years ago while shopping in a government department store in Beijing or Tianjin I spotted a tee-shirt on a mannequin in the sporting goods area with the following printed in English on the front, "PROPERTY OF THE U.S.A. ANTIQUE DEPT" [sic].

It would go well with your new kicks.
 
gnasher729 said:
Wrong on two accounts: Marketing isn't just advertising. Marketing is all possible ways to get the message about your product out to the public. Therefore, cheating on benchmarks _is_ marketing. And in Germany, "unfair competition" is a very broad term. Misinforming customers is just one form of unfair competition, and whether it happens in advertisements where it would be quite obvious or in a more underhanded form like Samsung's benchmark cheating doesn't really matter.

Second, they are using more than the CPU speed that is available to the customer in normal operation. They are using a CPU speed that is specifically reserved for applications named "GeekBench 3" or one of a dozen other names. If the phone cannot handle using what you call "full CPU speed" continuously, then using it in a benchmark is cheating.
Click to expand...

Wrong on two accounts: They are not the ones doing the benchmarks and they are not saying to anyone look at that benchmarks.

I don't know why it is such important that German law when this practice has been common since mid 90's with graphics and CPU benchmarks and no one has never said, look, they are illegal. Can you point a single reference to say that it would be illegal.

And two, they are not overclocking the CPU, and that speed can be reached with normal operation.

I don't know why don't accuse them just of being unethics and point the fingers but the illegal accusation is just ridiculous
 
The idea that this should be illegal is ridiculous, it's a free market, the last thing we need are laws about stuff like this - the unintended consequences would be HUGE and prevent companies from legitimately optimising any software. Trust me, laws about issues like this that Congress couldn't possible comprehend are the last thing you want.

It is, however, something the market needs to shame manufacturers for. I found it odd nVidia didn't cheat, they seem to have learned their lesson. In the good old days, they were the biggest cheaters of them all on PC benchmarks.
 
Zellio said:
Speaking of benchmark cheating, how about the '64 bit processor and twice the power' when you only have 1 gig ram, and most programs won't even be 64 bit in a long time?
Click to expand...

What proof do you have to your claim.?are u saying apple is misleading consumers with the A7 chip?
 
Last edited:
Oletros said:
And, actually, the phone is as fast as they say, they are not overclocking it, they are using the full CPU speed.
Click to expand...

Oletros said:
And two, they are not overclocking the CPU, and that speed can be reached with normal operation.
Click to expand...

Not really entirely true. Samsung actually allowed benchmark software to use a higher clock on the GPU of the Galaxy S4 that wasn't available for most other apps, even in games. http://www.anandtech.com/show/7192/update-on-gpu-optimizations-galaxy-s-4

philips said:
You simply have no understanding how the manufacturing works. You mixing up Samsung Semi and Samsung Electronics also suggests poor knowledge of the companies: unlike Apple, Samsung isn't vertically integrated company; there are many, loosely cooperating Samsungs.
Click to expand...

"Samsung Semi" is a division under Samsung Electronics the public company. More precisely, the semiconductor division of Samsung Electronics consists of various subdivisions and the System LSI subdivision is responsible for producing system chips. I'd expect someone to know these better before telling someone else they have "poor knowledge".

You're correct Samsung isn't vertically integrated like Apple. They follow the Japanese model more closely where each division will try to make a full business out of it instead of only serving other divisions and Samsung clients.
 
ikramerica said:
Originally Posted by NutsNGum View Post
Guy 1: My phone is faster than your phone!

Guy 2: I have a girlfriend.
Click to expand...
Guy 3: My girlfriend is faster than your girlfriend.
Click to expand...

Guys 1 & 2: We know.

----------
Applesexual2 said:
Apple "designs" CPUs in the same way you design your computer by buying an Intel CPU, an Nvidia GPU, Samsung dRAM, Samsung SSD and some power supply.

Just think of ordering a pizza. That's pretty much how Apple's "Designs" their CPUs. They can determine the clockspeeds and the GPU they want to use, but that's about it.
Click to expand...

No, since the A6 Apple has been *designing* it's own CPUs. The rest of the SOC is, indeed, off the shelf 'parts', but Apple designed the CPU and the parts of the SOC that integrate the CPU with those other components.

So, Apple "designs" CPUs in the same way that Intel "designs" CPUs. By designing them.

----------
magbarn said:
Is English not your primary language. All your quote basically says is that Samsung is flexible and not restrictive with their chip. designs. Not that Samsung designs their chips exclusively.
Click to expand...

Actually, that quote just says that Samsung chose the process in question because it reduces the feature size of chips produced using that method, and it doesn't add a lot of constraints on how chips must be laid out. You're certainly right, though, that it *doesn't* even remotely imply that Apple's CPU & SOC are Samsung designs as that fellow tried to claim.
 
For those who haven't read the article, here are Anand's closing comments. I'm not sahing what they did was alright, just pointing out that they aren't gaining a huge advatage with the cheats.

"The hilarious part of all of this is we’re still talking about small gains in performance. The impact on eour CPU tests is 0 - 5%, and somewhere south of 10% on our GPU benchmarks as far as we can tell. I can't stress enough that it would be far less painful for the OEMs to just stop this nonsense and instead demand better performance/power efficiency from their silicon vendors. Whether the OEMs choose to change or not however, we’ve seen how this story ends. We’re very much in the mid-1990s PC era in terms of mobile benchmarks. What follows next are application based tests and suites. Then comes the fun part of course. Intel, Qualcomm and Samsung are all involved in their own benchmarking efforts, many of which will come to light over the coming years. The problem will then quickly shift from gaming simple micro benchmarks to which “real world” tests are unfairly optimized which architectures. This should all sound very familiar. To borrow from Brian’s Galaxy Gear review (and BSG): “all this has happened before, and all of it will happen again.”
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back