Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Quick hold on to something as the tidal wave of tech spec geeks is about to happen!

Indeed. Don't even get that site. How did they post that entry on Sept 26?
"Posted by: admin on September 26, 2011 Under: Benchmarks"

Is their clock running late or what.
 
Has anyone else seen the rebuttal to the SunSpider Benchmark results?

http://www.bestsmartphone.com/2011/09/26/javascript-benchmarks/

41655.png


They fixed it. The phone still is still eating everyones lunch.

Update: We made a mistake in our original presentation of the SunSpider numbers and compared the iPhone 4S' 0.9.1 results to our existing database of 0.9.0 scores. We have since updated the graph to compare directly to our 0.9.1 numbers. The rest of the results are unaffected. I apologize for the confusion.
 
When you evaluate a system you have to evaluate it not just on the merits of the hardware but how well the software works with the hardware. Comparisons are fair game in my opinion.

Full multitasking is overkill for these mobile devices. And this is coming from someone who has used 2 android phones as my main phones for the last 2 years. Personally I think the service based system that Apple is using is far superior to letting tasks have free reign in the background.

My point is not "do we need full multitask on smartphone". I just point out the different RAM usage between two OS reflects the fact "they handle task differently. That's all.
 
My point is not "do we need full multitask on smartphone". I just point out the different RAM usage between two OS reflects the fact "they handle task differently. That's all.

Right but comparing their benchmark outcomes is fair. Its like saying a car has american parts and japanese parts. We can still take it to the dyno and see its power output curves and torque etc. Its still a car so we treat it as such. Just like these phones. They still have a web browser.

Unfair comparisons would be post deployment tweaks. Anandtech's results are all stock unmodified roms. I think its fair to compare from this baseline because 90% of users are running stock unmodified installations.
 
Image

They fixed it. The phone still is still eating everyones lunch.

This chart is misleading by being incomplete. Android phones get low score in this test because the stock browser in 2.* Android uses only single CPU core. That is not the case in Android 3.* or in third party browsers. Obviously, the stock browser will be updated in Ice Cream Sandwich (in two weeks). Here are the scores for Galaxy SII with two different browsers:

[Smartphone] Samsung Galaxy SII @ 1.2Ghz with Android 2.3.3 & Firefox Beta – 1370
[Smartphone] Samsung Galaxy SII @ 1.2Ghz with Android 2.3.3 & Opera Browser – 1600


Here SGSII destroys iPhone 4S which is not really surprising now that we know that A5 in iPhone 4S is clocked at 800MHz.
 
This chart is misleading by being incomplete. Android phones get low score in this test because the stock browser in 2.* Android uses only single CPU core. That is not the case in Android 3.* or in third party browsers. Obviously, the stock browser will be updated in Ice Cream Sandwich (in two weeks). Here are the scores for Galaxy SII with two different browsers:

[Smartphone] Samsung Galaxy SII @ 1.2Ghz with Android 2.3.3 & Firefox Beta – 1370
[Smartphone] Samsung Galaxy SII @ 1.2Ghz with Android 2.3.3 & Opera Browser – 1600


Here SGSII destroys iPhone 4S which is not really surprising now that we know that A5 in iPhone 4S is clocked at 800MHz.

Its fair because those are not the browsers that come with the phone. Are we browser benchmarking or phone benchmarking? It clearly says they use stock browsers with stock OS installations. I think that is a fair comparison. Comparing every browser is not very fair IMO. We care about out of box standard performance. If you start adding 3rd party software in then its a slippery slope. I overclocked and undervolted my last 2 android phones. We might as well test that too. You can't do it fairly unless you take the stock out of box browser and OS. Anadtech is doing it right.

Once you vary too many things it really clouds the results.

A better argument is that browser benchmarks are not very accurate when trying to determine overall system performance. I totally agree with that. Main issue being that javascript performance is not directly correlated with processor output due to it being a scripted language and different browsers using different interpreters and hacks to get things to work better.
 
Its fair because those are not the browsers that come with the phone. Are we browser benchmarking or phone benchmarking? It clearly says they use stock browsers with stock OS installations. I think that is a fair comparison. Comparing every browser is not very fair IMO. We care about out of box standard performance. If you start adding 3rd party software in then its a slippery slope. I overclocked and undervolted my last 2 android phones. We might as well test that too. You can't do it fairly unless you take the stock out of box browser and OS. Anadtech is doing it right.

A better argument is that browser benchmarks are not very accurate when trying to determine overall system performance. I totally agree with that.

Those test do have merits but then all these browsers are free :) Besides, the software is easily upgradable. SGSII will get ICS update in a few months and the score even for stock browser will improve drastically while iPhone's hardware obviously will still be the same.
 
iPhone 4s leave 64mb for developers to use, I was really hoping they can double this limitation after 16 months. Not to mention iOS 5 would use more ram than its predecessor.

Of course I know the developers can be smart enough to use the 64 mb efficiently. But this may cut the possibility of more powerful app, especially for image processing. By the way, you may see more memory warning and crashes using these photo taking apps as the result of better photos from the camera.

I own a galaxy s2 which drains battery in 6 - 8 hours without much use. So I really don't think better CPU means better phones, especially the CPU is not the bottleneck for many apps. Any graphics intensive work should be left to the GPU.
 
Wirelessly posted (Mozilla/5.0 (iPod; U; CPU iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5)

I'll most likely get a bias answer from most of those who choose to answer this since after all this site is meant for Apple.

I'll get straight to the chase. I've owned apple products from iPod classics to touch along with my MacBook. But I never got an iPhone. I was recently about to switch my old flip style phone to a smartphone. I've done a lot of research and came up with either the samsung galaxy s2 or HTC sensation XL (the dr dre endorsed) reason why I came up with sensation is because I love music and I'm a bit picky when it comes to sound quality. Although the HTC is endorsed by Dre Sound chip which to be honest all Dr Dre headphone products should only be worth $100 or so as their sound is not a mid ranged audiophone price. Just like most apple products, overpriced for limited stuff.

Anyways so I made up my mind and was about to buy the samsung galaxy s2 until the bestbuy dude told me that I should wait for iPhone 5 or 4S. So I waited, now after seeing this benchmark I was shocked that it beat the galaxy s2 by quite a bit. So question is should I buy the iPhone 4S? I mean it still doesn't make sense how could the iPhone 4s which is 200 or so ghz less than the 1.2ghz galaxy s2 and half the ram of S2. Have a higher benchmark? Is it the graphical chip?

Thanks a lot guys. I'm okay with bias answers as long as one can kinda backup their statement.

There is a separate benchmark for the GPUs. The reason you see an apparent discrepancy is that GHZ has very little to do with CPU performance and even less to do with system performance. The 4s is optimized hardware and software engineered to work together.

The S2 is a bunch of un-optimized hardware components selected by marketing to create spec sheet buzz with Android slapped on top.

Call me biased if you like, that does not change reality. The performance of the devices and the over 200 million satisfied iOS users speak for themselves.
 
Right but comparing their benchmark outcomes is fair. Its like saying a car has american parts and japanese parts. We can still take it to the dyno and see its power output curves and torque etc. Its still a car so we treat it as such. Just like these phones. They still have a web browser.

Unfair comparisons would be post deployment tweaks. Anandtech's results are all stock unmodified roms. I think its fair to compare from this baseline because 90% of users are running stock unmodified installations.

Again, you mislead the topic. I'm saying the reason of different RAM usage but you're talking about benchmark result.
 
Again, you mislead the topic. I'm saying the reason of different RAM usage but you're talking about benchmark result.

Your initial post was a claim that you can't compare the 3gs and the droid pro because of RAM consumption differences between ios and android. When you look at system usability(not talking about benchmarks now), those things are a factor. There differences does not affect usability comparison.

I am confused how a memory optimization deficiency makes comparing the two devices usability not allowed. Maybe I am misunderstanding.

Those test do have merits but then all these browsers are free :) Besides, the software is easily upgradable. SGSII will get ICS update in a few months and the score even for stock browser will improve drastically while iPhone's hardware obviously will still be the same.
Anandtech can't test software that has not been released. We can't hypothesize about possible performance gains. Also, just because a software component is free does not mean that it should be used when benchmarking. All kernel modifications for my HTC evo are free too.

Also when SGSII gets ICS its hardware will stay the same too. Not sure why you mentioned that. The SGSII is a great device. I actually debated between that and the iphone 4s.
 
Funniest thing in this chart is that it includes a tablet (apparentl to put a Samsung device on top?), but leaves off the iPad 2 (presumably to put a Samsung device on top?).

Weird stuff.

Not that funny, when you know the rationale for including it. To show the effect how the different OS versions improved the browser performance.

There's no difference between the that tablet and the Droid X2 CPU-wise, both are Tegra2 @ 1GHz. So, the the difference in numbers is the performance increase from using Android 3.0 compared to Android 2.2
 
Your initial post was a claim that you can't compare the 3gs and the droid pro because of RAM consumption differences between ios and android. When you look at system usability(not talking about benchmarks now), those things are a factor. There differences does not affect usability comparison.

I am confused how a memory optimization deficiency makes comparing the two devices usability not allowed. Maybe I am misunderstanding.


Anandtech can't test software that has not been released. We can't hypothesize about possible performance gains. Also, just because a software component is free does not mean that it should be used when benchmarking. All kernel modifications for my HTC evo are free too.

Also when SGSII gets ICS its hardware will stay the same too. Not sure why you mentioned that. The SGSII is a great device. I actually debated between that and the iphone 4s.

ICS is a result of merging of Android 2.* and 3.* (phone and tablet editions). It will support dual core. We can easily predict what ICS performance in browser will be because it's going to be the same (or better) browser as in Android 3.*. The reason I mentioned no hardware update for iPhone is simple: Safari in iOS 5 is already optimized for dual core CPU. There might be some incremental improvement there but nothing like doubling the number of cores. SGSII is in a different situation. While (in the context of this particular test) SGSII's CPU is much more powerful than A5 the bench score is distorted by the fact that the browser does not support dual cores.
 
Other than games what are the real use implications of better graphics processing? I don't really game much on my iPhone. I would probably rather have 1GB of RAM instead.
 
Your initial post was a claim that you can't compare the 3gs and the droid pro because of RAM consumption differences between ios and android. When you look at system usability(not talking about benchmarks now), those things are a factor. There differences does not affect usability comparison.

I am confused how a memory optimization deficiency makes comparing the two devices usability not allowed. Maybe I am misunderstanding.

I'm saying the reason which make difference. If one knows reason, he would know this kind of comparing is like comparing apple to orange.

BTW, when it comes to usability, actually more data store at RAM(high RAM usage) is better. Less RAM usage does not suggest memory optimization.
 
Last edited:
I'm saying the reason which make difference. If one knows reason, he would know this kind of comparing is like comparing apple to orange.

BTW, when it comes to usability, actually more data store at RAM(high RAM usage) is better. Less RAM usage does not suggest memory optimization.

You lost me here. Not sure what you are trying to get at. Actually this whole post confuses me.
 
ICS is a result of merging of Android 2.* and 3.* (phone and tablet editions). It will support dual core. We can easily predict what ICS performance in browser will be because it's going to be the same (or better) browser as in Android 3.*. The reason I mentioned no hardware update for iPhone is simple: Safari in iOS 5 is already optimized for dual core CPU. There might be some incremental improvement there but nothing like doubling the number of cores. SGSII is in a different situation. While (in the context of this particular test) SGSII's CPU is much more powerful than A5 the bench score is distorted by the fact that the browser does not support dual cores.

It might not even be that. It could be that Android 2.3's Javascript engine is an older version. Don't forget, Nitro in iOS 5 is the new Apple Javascript engine which was made available to Safari exclusively (no 3rd party apps or shortcutted apps on the home screen).

That's basically the whole problem with the benchmark, all it reflects is Javascript performance. If you spend all your time doodling around Javascript sites, then yes it's a fine benchmarks of which phone will be faster for you, but if you do more with your phone, it hardly indicates which hardware is faster.

Geekbench, while completely synthetic, would be a better hardware test, but alas, it doesn't seem to be available for Android, hence the lack of testing by Anand.
 
You lost me here. Not sure what you are trying to get at. Actually this whole post confuses me.

I guess that confusion comes from you don't know how OS works. So you compare RAM usage between 2 different OS and think it comes from "memory optimization deficiency" by one.

As matter of fact, OS do not optimize the RAM usage of an app. OS just try to allocate as much as app request, no more no less.
 
Last edited:
I guess that confusion comes from you don't know how OS works. So you compare RAM usage between 2 different OS and think it comes from "memory optimization deficiency" by one.

As matter of fact, OS do not optimize the RAM usage of an app. OS just try to allocate as much as app request, no more no less.

Actually, it's much more than that. There's tons of things the OS can do with free RAM that is much better than just letting it sit there unused. Take Linux for instance. Looking at a dev box at work right now configured with 14 GB of RAM, I see the following values :

Code:
# cat /proc/meminfo | egrep "MemTotal|MemFree|Cached"
MemTotal:     14369696 kB
MemFree:         97904 kB
Cached:        7186040 kB

So essentially, it has 100 MB of Free RAM. Linux is a pig right ? Look at the 3rd value... It's about half of the RAM in the server. What does it mean ? That's the file cache. Basically, Linux is using 7 GB of free RAM to cache files and thus speed up I/O by not having processes wait for slow disk/network access.

Now, some might say because the RAM isn't free, the OS is a pig. Others would realise the OS is smart and speeding up I/O by using unused ressources and thus making the whole experience better in the end. The file cache is also not really "used" RAM, the OS can deallocate older objects quickly and move the RAM to user space processes as allocations get requested, so it's a complete non-issue.

RAM usage is not an indicator of poor optimization, you are right about that. In fact, quite the contrary, RAM usage can be an optimization in and of itself.
 
Last edited:
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.