Virginia Tech Supercomputer - We're Still #3!

[MacRumors]

The latest numbers (PDF, page 53) (http://www.netlib.org/benchmark/performance.pdf) for the Virginia Tech PowerMac G5 Cluster, brings it up to 10.2 TFlops.

Last week's numbers (http://www.macrumors.com/pages/2003/10/20031028162457.shtml) were at 9.5 TFlops, moving the cluster up to 3rd place. The current #2 ranked computer sites at 13.8 TFlops.

[Nutzoids]

It would be nice to know if Microsoft has any computers on this list? Also what OS are they using 10.2 or 10.3?

[Ambrose Chapel]

Originally posted by Nutzoids
Also what OS are they using 10.2 or 10.3?

The machines are currently running stock Mac OS X 10.2.7. An audience member asked if they use Software Update. Varadarajan said no but that there are plans to Pantherize the system in the next few weeks. This will require an install and a recompile of some of the code.

http://www.macdevcenter.com/pub/a/mac/2003/10/29/osxcon_g5cluster.html

[Floop]

Varadarajan said ... that there are plans to Pantherize the system in the next few weeks.

Please tell me that they are not using any external Firewire Hard Discs using the Oxford 922 Bridge...

[MOM]

It looks like they would have to reach just under 80% efficiency to get the #2 spot. Anyone know if that is feasible? Either way, #3 in the world for just over $5 million means Apple is going to be selling a few more of these in the future.

[From Win to Mac]

Panther is waaaaay too buggy for them to upgrade. Probably Apple told them so, lol

[tny]

Originally posted by From Win to Mac
Panther is waaaaay too buggy for them to upgrade. Probably Apple told them so, lol

And I assume you've been using it regularly? I have noticed very few bugs so far (and I use it regularly, with - among other things - a LaCie 80 GB FireWire hard drive). Main problem I'm having is that every once in a while the screen saver tries to come on; must be a key binding I'm not aware of.

[Mr. Anderson]

Regardless of whether they go with Panther or not - they've solidly gained the #3 slot. That's damn good! :D

Now, if I could just get my own little G5 cluster......

D

[nagromme]

I take it that none of this matters for the Top 500 list? The earlier/slowest numbers were already submitted for that?

Real-world rank is more important anyway, but I wondered about the list. Some people talk as though the latest results will affect that list, even though I thought the deadline for submissions was long past.

[DeusOmnis]

More and more testing, I dont think they'll get to number 2 though.

[Chobit]

Originally posted by nagromme
I take it that none of this matters for the Top 500 list? The earlier/slowest numbers were already submitted for that?

Real-world rank is more important anyway, but I wondered about the list. Some people talk as though the latest results will affect that list, even though I thought the deadline for submissions was long past.

I've heard that they can update the score until November 18. Does anyone know if this is correct or not?

[stoid]

Yes, I was under the impression that November 17th was the final date for last minute speed tallies.

[Ambrose Chapel]

Originally posted by Chobit
I've heard that they can update the score until November 18. Does anyone know if this is correct or not?

i'm pretty sure they still have time to try and increase their score..can't remember the date, but that sound about right.

[PeteY48]

Originally posted by nagromme
I take it that none of this matters for the Top 500 list? The earlier/slowest numbers were already submitted for that?

Real-world rank is more important anyway, but I wondered about the list. Some people talk as though the latest results will affect that list, even though I thought the deadline for submissions was long past.

The system had to be operating and registered on Oct 1, but the school gets to optimize the system and report updated speeds all the way up to the conference which is around Nov. 20th.

[johnnowak]

Originally posted by tny
And I assume you've been using it regularly? I have noticed very few bugs so far (and I use it regularly, with - among other things - a LaCie 80 GB FireWire hard drive).

Ditto.

[danbirchall]

The scores up through the 9.555 teraflop one were done with only 2,112 CPU's. The newest one has the full 2,200 CPU's online.<p>

No, that doesn't account for all of the increase - adding those 44 machines would've gotten them up to around 9.9something teraflops, not 10.28.<p>

IF they're using the standard math libraries in 10.2.7...<p>

AND IF they're doing stuff that makes heavy use of AltiVec and vecLib...<p>

...THEN upgrading to Panther would send the scores through the roof. On my G5, XBench reports 2.6 gigaflops for "AltiVec Basic" and 2.18 gigaflops for "vecLib FFT" under 10.2.8, and 3.99 gigaflops/3.44 gigaflops respectively under 10.3. That's gains of 53% and 57% on those parts of the benchmark.<p>

[Mr. Anderson]

Originally posted by danbirchall
...THEN upgrading to Panther would send the scores through the roof. On my G5, XBench reports 2.6 gigaflops for "AltiVec Basic" and 2.18 gigaflops for "vecLib FFT" under 10.2.8, and 3.99 gigaflops/3.44 gigaflops respectively under 10.3. That's gains of 53% and 57% on those parts of the benchmark.<p>

Given a 50% gain we'd be talking 15 TF - that would put us in #2. Although getting 1100 machines upgraded would be quite painful in such a short amount of time.

D

[Ambrose Chapel]

Originally posted by Mr. A
Although getting 1100 machines upgraded would be quite painful in such a short amount of time.

well...there's a lot of us here on MR...how badly do we want VA Tech to be number 2?
:D

[MOM]

Originally posted by Mr. Anderson
Given a 50% gain we'd be talking 15 TF - that would put us in #2. Although getting 1100 machines upgraded would be quite painful in such a short amount of time.

D


15 TF would be pushing very close to 100% efficiency. I don't think things scale like that. There is overhead and latency with bringing so many computers together.

My bet is they have an efficient method of upgrading the computers to panther.

[P-Worm]

That and they have a lot of computing power to install the operating system. 10 Teraflops in fact.

P-Worm

[Viv]

Originally posted by Ambrose Chapel
well...there's a lot of us here on MR...how badly do we want VA Tech to be number 2?
:D

How many Mac engineers do they need?

Put me down for one.

By the by, the managers of the machines below the VA tec cluster must be fielding some interesting questions from thier bosses!

For instance "they are at number two for 5 million bucks!" "so were are we for $XXXXX

Viv

[Frohickey]

Upgrading 1100 G5s to Panther?

Thats a lot of trouble just to be able to surf for porn quicker. :D :p

[sharky2313]

when will this cluster begin folding for Mac Rumors? We could use a little push :)

[Sol]

Even without FireWire drives or FileVault encription, Panther would be a high risk. It is simply a 1.0 release and thus too unpredictable for such a mission critical task as this.

If I was a part of this project my attitude would be that over 10 terraflops is a great result for a cluster that cost under 10 million dollars. Virginia Tech should start running their science programs on Big Mac and work on optimisations between now and the time to upgrade the hardware.

[mvc]

Originally posted by Frohickey
Upgrading 1100 G5s to Panther?

Thats a lot of trouble just to be able to surf for porn quicker. :D :p

They don't have to surf, with this amount of grunt they can render it in real time!

Sure beats beach volleyball :D

[stingerman]

They can add more G5's, they should get another 500.

[Rocketman]

Originally posted by MOM
15 TF would be pushing very close to 100% efficiency. I don't think things scale like that. There is overhead and latency with bringing so many computers together.

My bet is they have an efficient method of upgrading the computers to panther.

About 10 years ago or so Harvey Mudd College used to update all the Mac IICI's over the network.

MacOS server does remote admin and update.

I suspect updating to panther is as simple as debugging a single mac, making an image and propogating it to all units essentially instantly.

Of course if there is a bug one has to downgrade :)

I doubt there is a Panther bug of note to simply running a benchmark since they do not use filevault or FW800 at all.

Rocketman

[mattevil]

you know i live only about 20-30 miles away from va tech and i had to go to macrumors to fiind out about this! funny thing is i was talking to one of their techies couple of years ago and they were questioning why i would use a mac nowadays!what a turnaround!

[dho]

Dho wishes he was a virginian :(

[nagromme]

Originally posted by PeteY48
The system had to be operating and registered on Oct 1, but the school gets to optimize the system and report updated speeds all the way up to the conference which is around Nov. 20th.

Thanks for the clarification. That explains it :)

[iEric]

And what does this supercomputer do?

[Sol]

Originally posted by iEric
And what does this supercomputer do?

Apparently it tries to crawl into top-ten charts.

Seriously, as a long-time Mac user I would get a lot more satisfaction from learning that Big Mac was used to find a cure for AIDS. Benchmarks are not an end in themselves. It is about time Virginia Tech did something useful with Big Mac.

[solvs]

If it's number 2 or 3, it will. I'm sure the research facility will have plenty to do if the #'s look good.

[ITR 81]

I say even if they don't make #2 by the 20th I believe they will be very close and they probably will hit #2 unoffically by the end of Nov.

I think if they can get all the coding done to use Panther before the 20th then I say they will take the #2 spot.

I'm just wondering doing this same cluster with next G5's pushin 3Ghz clock speeds? I figure that in next couple yrs Apple will have the fastest cluster if folks use it in way Va Tech has.

[Sol]

Originally posted by ITR 81
I'm just wondering doing this same cluster with next G5's pushin 3Ghz clock speeds?

I am sure that someone, somewhere will read about Big Mac and think "Hey, if they can do that with $5 million worth of PowerMacs imagine what I can do with this blank cheque!" Needless to say, bigger Big Macs are inevitable and I suspect that most institutions would do the sensible thing and wait for G5 XServes to become available.

[Sun Baked]

Should be interesting to see what IBM's Quad PPC970 Blade Servers are capable of.

[loneAzdgari]

Its sort of ironic to think that computers have returned to room sized machines. In 20 years we'll be reminising about how our mobile phones contain more power than this machine. ;)

[MadMan]

Originally posted by Floop
Please tell me that they are not using any external Firewire Hard Discs using the Oxford 922 Bridge...

It's not just the 922 chip... My 250GB Maxtor FW400 drive hit the same problem. I was lucky... Rebooting into OS 9 (I had inited the drive with OS 9 Drivers) allowed my to repair it and get my data off. Very scary!!!!

MM

[zuffen]

One thing to remember the machine cluster cost 5million, but the building and support infrastructure probably cost just as much if not more than that. And then you have payroll, which could easily be 1 million a year.

Iwould like to be around when the scrap the thing in 5 years, think I could buy it for cheap?

[mattevil]

they still have g3 towers on campus so i doubt they'll scrap them.

[Rincewind42]

Originally posted by danbirchall
AND IF they're doing stuff that makes heavy use of AltiVec and vecLib...

Unfortunately they can't use altivec for the benchmark - to qualify you need to use at least 64-bit floating point, which altivec doesn't do. If they were using altivec, they could be getting nearly triple the performance (the two floating point units doing 2 ops per cycle, plus the altivec unit doing 8 ops per cycle, tripling the number of ops that can be done per cycle).

Altivec is great for a lot of things, but unfortunately not for 64-bit floating point math.

[sethypoo]

This is great! If only they could get it up to 12.....then so and so forth. Could you imagine a 5000 node G5 setup? Besides causing a new El Nino, we'd get awesome TF!

[SiliconAddict]

Originally posted by Nutzoids
It would be nice to know if Microsoft has any computers on this list?

Why? Microsoft doesn't make hardware. :rolleyes:

[ghutchis]

Seriously, as a long-time Mac user I would get a lot more satisfaction from learning that Big Mac was used to find a cure for AIDS. Benchmarks are not an end in themselves. It is about time Virginia Tech did something useful with Big Mac.


So there's a few things to keep in mind. One is that there's probably something in the grant that says they're going for this-and-such level of performance, so they need to benchmark in order to fulfill that part of the grant.

VT is hiring lots of new faculty members to keep this thing fed. So the publicity also helps them get better applicants--I know at least two friends applying.

I can understand the "frustration" sitting on the sidelines. But I can tell you that even with a 30-40 node Linux cluster setup that it takes some time to get things up to speed and stable. A recent cluster here took about a month to work out the kinks in the network drivers. Some of this time for benchmarking would be needed anyway before they open it to the campus. It allows them to improve uptime and reliability as well as overall performance. Since they'll effectively charge for the CPU time, it's worth it to VT to spend a few weeks optimizing!

I'd guess they'll "open for business" quietly in a few more weeks and maybe more publicly in Dec./Jan. And I'm sure they'll get some innovative research projects--it's a lot of new horsepower!

[SiliconAddict]

I’m actually somewhat amazed they are even using OSX for the OS on these systems. I have to imagine a certain % of the CPU is being used for the GUI, even if it is a small % since the GPU handles a good % of the GUI. Going with something like one of the various flavors of Linux, that is already optimized for 64-bit computing, but most likely not for the 970, I have to imagine a pure command line environment with no GUI would be able to squeeze a tad more performance out of these systems.

[zaphoyd]

Mac OS X can run pure command line if you like.

[Rincewind42]

Originally posted by SiliconAddict
I’m actually somewhat amazed they are even using OSX for the OS on these systems. I have to imagine a certain % of the CPU is being used for the GUI, even if it is a small % since the GPU handles a good % of the GUI. Going with something like one of the various flavors of Linux, that is already optimized for 64-bit computing, but most likely not for the 970, I have to imagine a pure command line environment with no GUI would be able to squeeze a tad more performance out of these systems.

The GUI environment only requires CPU time if there is something that it actually needs to do - I've literally seen the CPU usage say 0% at times on a PBG4 400 when I just wasn't doing anything at all (yes, a pre-quartz extreme machine). That, and if the machines are running headless, I'm pretty certain OS X won't try to draw anything. As for linux, there isn't a distribution that can control the G5's cooling systems yet (I imagine it's pretty loud in there now, imagine with 9900 fans going full blast too). And they are planning on moving up to Panther anyway, so it doesn't look like linux is an option in their opinion.

But wouldn't it be ironic if someone forgot to set the Energy Saver performance setting to Highest when they setup the machines?

[SiliconAddict]

Originally posted by Rincewind42


But wouldn't it be ironic if someone forgot to set the Energy Saver performance setting to Highest when they setup the machines?

:eek: They jump from #3 to number #2 with the click of a mouse. That would be funny as heck. :D

[xone]

Originally posted by zuffen
One thing to remember the machine cluster cost 5million, but the building and support infrastructure probably cost just as much if not more than that. And then you have payroll, which could easily be 1 million a year.

I'm pretty sure that the $5 million includes the infrastructure too.