PDA

View Full Version : More G5 Benchmarks (NASA)




MacRumors
Jul 4, 2003, 11:19 PM
Several have noted these benchmarks (http://members.cox.net/craig.hunter/g5/) from an individual who works at NASA.



wizard
Jul 4, 2003, 11:47 PM
Its a shame that this testing wasn't done with up to date compilers. It appears that they just ran existing binaries.

On the other hand it does reinforce some things, one of which is that the G5 isn't all that its cracked up to be but isn't a slouch either. Apparently this is with old code that was written for the G4, it would be very interesting to see how everything runs when compiled against a modern compiler targetting the G5.

Thanks
Dave

nagromme
Jul 4, 2003, 11:55 PM
Looks like well-design tests, for their purpose. Results use only a single CPU, but they note the G5s dual performance in the text: very nearly double, which is good!

I suppose they used single CPUs for fairness in comparison to Pentiums, which don't exist in duals? Fairness, of course, meaning evaluating the chip's capabilities for their very specific needs. Not fairness in terms of benchmarks for marketing, as they make clear.

They do mention that they did no optimisations for the G5, and that better compilers might give the G5 a boost in future tests. Encouraging, since the G5 did well even so!

maradong
Jul 5, 2003, 03:20 AM
well they are really good for apple...
the g5 2ghz single cpu is 22% slower than the 3200 intel with fsb 800. well, but if the new compilers come, with optimisations the difference is only 2%....
a dual g5 2ghz is the fastest system aviable. because the intel machines have only one bus per processor, while the ibm chip has got one for each. that is significantly faster in dual cpu mode...
peace!

Blackcat
Jul 5, 2003, 05:50 AM
Interesting!

If I'm reading this right, a single 2GHz G5 is nearly 50% faster than a 1.25GHz G4 in MFLOPS, and 25% faster than a 2GHz P4.

Impressive!

rjstanford
Jul 5, 2003, 09:36 AM
One thing that struck me as interesting:Jet3D vector benchmark results are presented in Figures 3 and 4. Again, note that the vector benchmark does not include P4 systems because the AltiVec instruction set is only available on G4 and G5 systems. Consistent with earlier Jet3D tests, the vector version of Jet3D runs an order of magnitude faster than the scalar version (speedups of 10X-13X are typical). In this particular test, the raw vector performance of the G5 is impressive at 2755 MFLOPS, but a look at the normalized levels reveals that the G5 is nearly identical to the G4 in terms of vector performance per clock cycle. Thus, the increased raw vector performance of the G5 is largely due to its higher clock speed.There was a lot of speculation that the Alitvec unit was bus-limited in the G4. I'm not familiar with the test in question, but this should certainly be an interesting point, no? Is anyone more familiar with the code to know if it would have been expected to benefit from the faster bus?

-Richard

allpar
Jul 5, 2003, 10:13 AM
So then a 1.6 G5 would not be much faster, if at all faster, than a dual 1.256?

Originally posted by Blackcat
Interesting!

If I'm reading this right, a single 2GHz G5 is nearly 50% faster than a 1.25GHz G4 in MFLOPS, and 25% faster than a 2GHz P4.

Impressive!

Blackcat
Jul 5, 2003, 10:23 AM
Originally posted by allpar
So then a 1.6 G5 would not be much faster, if at all faster, than a dual 1.256?

Prolly not, but if a SINGLE 1.6 is the same speed as a DUAL 1.25, that's still impressive, no?

I'm no math whizz, but I think a 1.6 GHz G5 is 36% faster than a 1.25 GHz G4 in this test.

Heltik
Jul 5, 2003, 10:37 AM
Even though the margins are quite small for the benchmarks used, we have to consider that the Pentium did not use Vector processing.

For applications such as Photoshop/Final Cut/DVD authoring/Cd ripping, etc, surely we can expect the G5 to be leagues ahead, therefore. In fact, just about everything I use my Mac for on a daily basis.

scan300
Jul 5, 2003, 10:41 AM
Originally posted by Blackcat
Prolly not, but if a SINGLE 1.6 is the same speed as a DUAL 1.25, that's still impressive, no?

I'm no math whizz, but I think a 1.6 GHz G5 is 36% faster than a 1.25 GHz G4 in this test.

Keep in mind that these benchmarks test floating point and vector computational performance only. The programs only use 1 Mb of ram. They don't take into account system bandwidth.

So when evaluating the 1.6 GHz G5, the vector figures may not be much better than the G4, the floating point an improvement, but the data throughput on the bus will be significantly better than the G4.

maradong
Jul 5, 2003, 10:49 AM
Originally posted by scan300
Keep in mind that these benchmarks test floating point and vector computational performance only. The programs only use 1 Mb of ram. They don't take into account system bandwidth.

So when evaluating the 1.6 GHz G5, the vector figures may not be much better than the G4, the floating point an improvement, but the data throughput on the bus will be significantly better than the G4.
exactly, and that is much more important than raw gflops values.

Freg3000
Jul 5, 2003, 11:13 AM
Isn't there a thread on this already?

http://forums.macrumors.com/showthread.php?threadid=31163

Anyway, the results look pretty nice. :)

nuckinfutz
Jul 5, 2003, 02:45 PM
What impressed me the most was not the Altivec. Jon "Hannibal" Stokes from Arstechnica pretty much called the Altivec preformance correctly on his breakdown of the 970. I expect that the probably PPC 980 will shore up Altivec even better(adding it more cleanly to the design) resulting in good performance.

What we should be focusing on is the incredible FPU preformance on non Optimized Code. Recompile that code to take advantage of the G5's prodigious FPU capability and you will see some nice Real World gains.

allpar
Jul 5, 2003, 03:59 PM
...so the extra $300 to buy a single 1.6 is well worth it over the Dual 1.25 ... ? Just confirming ... the duals seem like a good deal ... I don't figure I'll ever need PCI-X...

aarond12
Jul 5, 2003, 08:48 PM
Did anyone notice the author's comment that enabling both processors in the G5 system made the G5 system nearly twice as fast as the Pentium 4 2.66GHz system?

Sayer
Jul 5, 2003, 08:58 PM
The G5-updated compilers will do several things to help boost the performance:

* The G5 has a differnet pipleline and it needs to be properly filled for efficient use of CPU resources. The newer compiler will do this.

* The OS does not have all the tweaks and changes Apple will most likely be putting into Panther (if Smeagol performed as well as Panther on a G5 Apple would lose the $129 upgrade since many people would stick with Smeagol with the same performance as Panther and save the money).

* The G5 works on instructions in groups, if the groups are not filled up optimally the G5 will not run optimally. The newer compiler and tools will properly handle the G5's quirks here.

I continue to be amazed at people who see a Mac finally on an equal footing to a top-end PeeCee and whine that its not FASTER than the absolute top-end PeeCee.

Apple essentially erased the only compelling hardware reason to not buy an Mac with the G5. It has every Wintel buzzword and trumps them (1 GHz busses, FireWire 800, optical audio I/O built in).

Now its just up to the software to step up and remove another reason NOT to use a Mac. Do you think Apple's X11 and Ports Manager software is meant to compete with XFree86 and the Darwin community or to get more big name Linux software on the Mac with an officially-sanctioned cross-grade option?

Mercury
Jul 7, 2003, 12:11 AM
Yay! Finally, some independent testing of the new units. It seems that finally we have the efficiency and the horsepower to brag to our PC friends.

Now, another $2200 to get top of the line with 23" HD...

ibookin'
Jul 8, 2003, 12:56 AM
Originally posted by Mercury
Yay! Finally, some independent testing of the new units. It seems that finally we have the efficiency and the horsepower to brag to our PC friends.

Well they'll just tell us how our machines are too expensive. :D

iriejedi
Jul 9, 2003, 10:55 AM
Maybe I've missed some of the threads that I'm requesting.... but show us some stats that matter!

How fast will games play?

How much will the DP 2gig benefit a gamer vs a 1.8gig single brain?

I use photoshop so I'm going dual processor - but I'm a soap box preacher trying to convert the UNSAVED PC users. How can I sell the slower models to people who play game instead of sequencing DNA without key data?

Iriejedi

nighthawk
Jul 10, 2003, 01:40 PM
Also note that the Pentiums were running Red Hat Linux, and not Windows... this actually places the P4s with a overall more effecient OS than OSX (but equal standing in low-level features). In my opinion, the scores are actually more impressive than they appear. Remember that Apple tests and the other tests compared the G5 to Windows OS machines.

g3ski
Jul 10, 2003, 01:47 PM
Originally posted by iriejedi
How fast will games play?

How much will the DP 2gig benefit a gamer vs a 1.8gig single brain?

Iriejedi

The 970 should be pretty competitive on the games front with the 1ghz bus and fast clock speeds, and a Radeon 9800 Pro.

But we will have to wait for some real benchmarks, until then it's just specualtion. Maybe they will be demoing the Towers at MWCP, possilby even an nVidia or ATI gaming station showing them off.

Seems like UT 2003 would be a great test.

Of course only a few games are optimized for dual processors...so Black and White should fly, but UT2003 will only see slightly more than 200Mhz bump between the 1.8 and the dual 2.0.

nighthawk
Jul 10, 2003, 01:55 PM
Originally posted by iriejedi
Maybe I've missed some of the threads that I'm requesting.... but show us some stats that matter!

How fast will games play?

How much will the DP 2gig benefit a gamer vs a 1.8gig single brain?

I use photoshop so I'm going dual processor - but I'm a soap box preacher trying to convert the UNSAVED PC users. How can I sell the slower models to people who play game instead of sequencing DNA without key data?

Iriejedi

First of all, in order to take advantage of a dual CPU, the software must be multi-threaded. Some of the top end games may start recoding for duals, but in the PC world, few end-user machines are dual. The only option (other than Xeon) are the Althon MP which are much more expensive and slower per chip than their XP counterparts.

I was at WWDC, and they had an entire group of computers just for gaming... at least 36. They were playing Unreal and Medal of Honor. The image quality and fluid motion were incredible... and best of all, they were only running on the G4s.

When I tested the G5's in the lab with Carbon Fractal Altivec, the dual 2.0ghz scored 12.4 GFLOPS at it's highest score (default settings).

Don't worry, the G5 will blow away the gamers with horsepower to spare.

allpar
Jul 10, 2003, 02:35 PM
How'd the dual G4 do in your test? I'm really looking hard for comparative benchmarks...

Flynnstone
Aug 28, 2003, 10:08 PM
Couple interesting bits:
The memory footprint was 1 meg. So on the G4 is probably all resided in L3 cache. So perhaps it wasn't beating up system bus!
So a dual G4 should be twice as fast.
The G5 doesn't have L3 cache, so it was likely using its system bus. Interesting the running on dual processors was twice as fast. Perhaps the lack of L3 cache is not an issue.

I like to see the person rerun the test on:
-G5 using the optimized compiler
- P4 Xeon (like Apple used for comparison) in single and dual configuration.
- P4 (non-Xeon) at 3.2 Ghz
- Athlon 3200
- Opterons Duals
- Athlon 64

jamesbsd
Aug 30, 2003, 01:59 PM
One thing which seems to have been missed in the discussion is that the problem being solved runs 10-13X faster when using code which accesses the Altivec unit. The P4 was not tested using this version of the code because the P4 is missing an Altivec unit. :)

One way to interpret these results, therefore, is: the G5 is the fastest computer for solving this problem, and only the G4 machines (with Altivec) were close.

Early posters seemed discouraged by these results, but I found them very encouraging, given that the G5 has very specific compiler requirements that couldn't be met when the Jet3D program was compiled. Presumably the test could be run again using IBM's recently released xlf compiler. There is reason to believe that very significant (40%) speed improvements may be forthcoming with a more G5-aware OS X (Panther), so stacking these two things together, it's quite likely that rerunning this simulation with xlf and Panther would demonstrate the G5's superiority vs. the P4 (without using Altivec...when using Altivec, it already isn't close) at higher GHz (3.2+?).

Benchmarking the G5 on code that doesn't utilize the VMX unit is really very misleading, as far as expectations of real world performance are concerned. Code that runs on the G5 *will* be written to use Altivec/VMX in the real world, and "testing" the G5 while the VMX unit lays idle can't be a good real-world test. It would be like benchmarking a Pentium on Mandelbrot fractals without compiling any code for the FPU--this would be very misleading. A benchmark which ignores large parts of a processor's real abilities is never a good benchmark, and this is part of my complaint about running the SPEC code to test machines that have vector processing units. The SPEC code ignores the VMX/Altivec units, a situation which no well-written real world code would ever do, and SPEC is therefore a very poor benchmark for processors like the G4 and G5 which have outstanding vector processing units.

All benchmarks need to be read with a grain of salt and a lot of knowledge. Without the grain of salt and only a little knowledge, benchmarking can be more dangerous than useful.

blackhorse95
Nov 10, 2003, 10:47 PM
I have only recently joined the ranks of Mac owners. I did this because I wanted to see for myself if all the smug bragging that I had seen and heard from Mac users was founded in reality. I had always assumed that it was just a huge load of B.S. Admittedly, I started out my mac experience at the lower end of the performance pool; a G4 400 mhz. However, as I begin to learn the workings of the mac architecture and OS X, I finally feel almost qualified to post some unbiased remarks concerning the performance of mac hardware vs. x86 pc hardware.

I also read the nasa performance ratings that are mentioned here; however, I did not get the same impression that most of the previous posters have. The bottom line that I got from the nasa benchmarking was that the 2 GHz G5 was roughly equilavent to a P4 2.6 GHz. I was somewhat surprised at this, since before this I had been seeing Apple G5 advertisements claiming the G5 was the fastest pc on the planet. I had assumed that the G5 was actually much faster than it tested out by NASA.

Anyway, what we all need to do is to quit speculating and assuming results that have not been independently and fairly tested. G5 processors MAY show higher performance when benchmarks are recompiled for this architecture, but until we actually see this demonstrated, it is extremely arrogant to assume so. Also, do not always make assumptions that place higher performance on Mac hardware and lower performance on X86 hardware. It is assumed here that X86 machines perform better running linux than Windows XP. Why would anyone assume this? While linux is at its core a unix derrivative, as is OS X; the comparison is hardly apples to apples. Pun intended. If we wanted to have a fair test between a P4 and a G5, we would use linux as the OS for BOTH machines. Using OS X gives a clear advantage to the G5. Why not see the results of the P4 running XP? I hope everyone here realizes that the Apple G5 tests and the associated publicity was entirely a marketing tool used by Apple to promote the G5. Do not get me wrong, I applaud Apple for the G5. I believe it is a great piece of hardware. But realistically I believe the hardware performance results to be pretty close to what was reported in the NASA test.

It seems like many Mac fans are obsessed with patting themselves on the back and socializing in a mutual appreciation society. I find this entirely similar to pro sports fans who watch their teams play on TV and say to each other how "we" sure kicked their ass Monday night!!!

I hope to continue to learn more about my Mac and OS X unix. I doubt that I will ever abandon my X86 systems in favor of them. So far, I see a nice platform with no real advantage. I would encourage everyone to take a first hand look at the "other team" before dismissing it entirely.

legion
Nov 11, 2003, 03:33 AM
Totally agree with you. I find the "self congratulatory" attitude to be a hindrance to the Apple/Mac platform realizing its potentional... and that's just it; at the heart of the Apple/Mac machine is tremendous potentional, it's just poorly realized and poorly marketed. If most people involved with Macs would get beyond the "whining" stage of anti-Mac consipiracies and "we're the best at ..." mentality and instead just produce great results, the platform and its merits would speak for itself.

Also, realizing that computers are tools (and not surrogate mothers, fathers, lovers, children, and best friends), there would be the epiphany that there are different tools for different jobs and that not one is inherently bad.

Some how along the way, these things have been lost and generations have been brainwashed by marketing departments.

DrBoar
Nov 12, 2003, 04:35 AM
With IBM linux blade servers soon using both IBM 970 AKA G5 and Intel Xeon we will have more valid comparisons of how well the G5 stacks up against the competition.

There still is the problem of bad programing and compiling but here there are less vested interests. I do not think anyone expect Quiktime to perform better in Windows than OS X or some applications ported from windows to run better in X than windows. Linux ought to be more "agnostic".

And the bottom line is if it feels fast it is fast. With small subsets of tests anything can be proven Client speed RC5-72 G4 1.25 MHz 13millon keys/s a Pentium 4 at 3.2 GHz only score 4.5 million and the AMD Opteron 8 million. This only proves that the G4 is way faster than any Intel CPU in this particular applcation thanks to AltiVec. In most application the Intel stomp on the G4.

bousozoku
Nov 12, 2003, 11:25 AM
Considering some of the idiots who work for NASA here in Floriduh, it's a wonder they get anything to compile at all. I think they're just glad to see it work and don't worry a whole lot about optimisation. That's where Apple's version of gcc, prior to version 3.3 has always broken down. It was terrible at optimising code and the PowerPC line is not good at handling bad code.

That said, I don't wish to try the "other team" at home. I've used it extensively for work at various places. Even the latest Windows XP is more stressful than Mac OS X 10.2 and above. With Mac OS X, though, many developers have had to discard their old "truths" because they just don't work anymore. I've seen several products which have become much slower on Mac OS X because they thought that they could simply add a Carbon resource file and re-compile, as they've been told could work. Of course, they didn't read through Apple's skimpy documentation to find that it doesn't work well to do that.

I'm sure Apple people working with Adobe and Macromedia find themselves frustrated each time.

At least, the new developers don't have any System 6 legacy to drag Mac OS X down.