Details of Va Tech Supercomputer Project

[MacRumors]

MacDevCenter provides detailed notes (http://www.macdevcenter.com/pub/a/mac/2003/10/29/osxcon_g5cluster.html) from a talk from Dr. Srinidhi Varadarajan about the Virginia Tech PowerMac G5 Cluster. (Presently ranked at #3 worldwide (http://www.macrumors.com/pages/2003/10/20031028162457.shtml))

According to the talk, the timeframe on the entire project was relatively short, with initial plans starting in March 2003. Initial negotiations were begun with Dell, IBM and AMD, but costs remained too high. In fact, the first Virginia Tech heard of the new PowerMac G5s was on their announcement day at WWDC (June 23rd).

Varadarajan met with Apple within the week and the deal was made. (Of interest, the purchase was actually made through the Apple Store.) Initial machines came with the stock Mac OS X 10.2.7, with plans to upgrade to Panther shortly.

Final plans for the cluster will be made freely available, and requests have already been coming in. Varadarjan says, "Expect to see a lot more G5 clusters."

Many more details available at MacDevCenter (http://www.macdevcenter.com/pub/a/mac/2003/10/29/osxcon_g5cluster.html).

[Doctor Q]

Dr. Srinidhi Varadarajan bought his supercomputer the way I buy computers: ask all your friends for advice, do some research into specs, look around for good prices, and then buy the best Mac you can afford. I'm glad he didn't have to cut out 1,100 product codes from the shipping cartons and fill out 1,100 rebate forms!

[kb9000]

Interesting, I would have guessed they had talked to Apple prior to the release of the G5s.

[Mudbug]

man, i would have liked to of seen the auto-generated invoice from the Apple Store for that...

woulda been fun to hold in your hand, just for giggles.

1100 - unreal :)

[dongmin]

Originally posted by Mudbug
man, i would have liked to of seen the auto-generated invoice from the Apple Store for that...

woulda been fun to hold in your hand, just for giggles.

1100 - unreal :)

uhh, you mean one of these:

[dongmin]

Varadarajan reported that "our latest numbers are 9.555 tera and we still have more tricks left. We are hoping for another 10 percent boost to become the first academic machine to cross 10 tera. The last ratings put us at number three worldwide." sweet! 10.5 Tflops!

[ITR 81]

Was that invoice with edu discount applied? Or did they just pay full price and Apple gave them sortof a edu rebate check?

Can you imagine when someone copies this and makes a G5 cluster with new 3GHz coming out next yr!!

But that really explains the reason for awhile no one could get G5 2GHz since most them went to Va Tech.

[punter]

there's an interview with him at
http://macslash.org/article.pl?sid=03/10/28/2357235&mode=thread

direct quote: "each processor can deliver 2 DP unites * 2 flops/cycle = 8GFlops. That's more than one Cray X1 Node. In a desktop. (*****.)"

I bleeted out the profanity. I assume he means "golly that's amazing", rather then "that's the performance I'd expect from excrement.

[MacFan26]

That's so awesome! That's really not that expensive, we could all afford 132,000 a month to have our own cluster right? :D

[dho]

muy interesante (sorry doing spanish hw)

I find that surprising that they were willing to swith to the g5 on such short notice
not that I mind

AT ALL :)

[Powerbook G5]

I want a cluster, maybe then I can get my papers typed quicker...look out Word, I'm sending 10.5 tera your way!

[dracoleb]

Hey only $132,089.56 per month.
Maybe I should pick one up. :D :p :D

[Blaaze]

Originally posted by dongmin
uhh, you mean one of these:

can't forget the edu discount :D

[coumerelli]

Originally posted by dracoleb
Hey only $132,089.56 per month.
Maybe I should pick one up. :D :p :D

and don't forget...6 months same as cash!

It really makes my 'check email...look at mac rumors...listen to iTunes...check email again...chat' a little underwhelming for such raw power.

[Phil Of Mac]

Originally posted by Macrumors
Final plans for the cluster will be made freely available, and requests have already been coming in. Varadarjan says, "Expect to see a lot more G5 clusters."

They should take all the G5 clusters and connect them together. I will gladly donate my dorm room for this project.

One Cluster to rule them all, One Cluster to find them, One Cluster to bring them all and in the darkness bind them!

Originally posted by ITR 81
But that really explains the reason for awhile no one could get G5 2GHz since most them went to Va Tech.

Actually, a vast minority of them went to Virginia Tech. If 1100 was most of Apple's Power Mac sales, they wouldn't be a Fortune 500 corporation.

[Doctor Q]

Did he buy AppleCare?

[whocares]

Do they make a portable one of these?

I'm often on the go and need portability, but will not sacrifice permformance. What's the battery life like?

[stcanard]

Originally posted by Doctor Q
Did he buy AppleCare?

Of course.

That's why the long wait for replacement LCD's!

J/K

:p

[sethypoo]

What happened to all those keyboards and mice? 1100 hundred of each.....

:) :rolleyes: :D

[ITR 81]

I would figure they kept a few on hand but the others probably got donated or maybe they sold them at cheap price.

[foniks2020]

Originally posted by ITR 81
I would figure they kept a few on hand but the others probably got donated or maybe they sold them at cheap price.

EBAY anyone?

[foniks2020]

Seriously... do they run SETI during off hours... if ther are any?

AND what are their numbers like?

[sushi]

Originally posted by sethypoo
What happened to all those keyboards and mice? 1100 hundred of each.....

:) :rolleyes: :D
Maybe they kept them.

When new Dual 3Ghz versions come out, they could merely swap new Macs for the old ones.

Then sell off the old Macs, with keyboards and mice to offset the cost of the new system.

Afterall, in a couple of years, Dual 2 GHz systems will be slow.

Who knows?! Just an idea.

Sushi

[jayscheuerle]

And he hasn't even upgraded to Panther yet...

He may get that extra 10?, eh?

[SoonToGetAMac]

Originally posted by jayscheuerle
And he hasn't even upgraded to Panther yet...

He may get that extra 10?, eh? Hey, if you have to pay $19.95*1100 to upgrade to Panther...:D

[gwuMACaddict]

Originally posted by Macrumors
Final plans for the cluster will be made freely available, and requests have already been coming in. Varadarjan says, "Expect to see a lot more G5 clusters."


FANTASTIC! and why not? turns out it's fast and cheaper. this is going to do a lot of good things for apple and steve-o

[arn]

Some great quotes:

http://www.wired.com/news/mac/0,2125,61005,00.html


Afterward, Derek Bastille of the Arctic Region Supercomputing Center in Fairbanks, Alaska, said he was blindsided by Varadarajan's achievement.

"It came completely out of nowhere," he said. "I'd never heard of him. If it doesn't have the Cray name on it, no one takes it seriously. It's incredible."

Bastille said his center had just spent $30 million and two and a half years building a pair of IBM-based clusters.

Another member of the audience, an unidentified representative from Lawrence Livermore National Laboratory, said the Big Mac project had seized the attention of the supercomputing world. (The lab's 2,300-node Linux cluster has fallen to No. 4 in the supercomputer rankings.)

"Needless to say, you're being watched," he said to Varadarajan.

During his talk, Varadarajan was treated like a nerd superhero. He received three enthusiastic rounds of applause -- the last one a standing ovation -- from the 100 or so Mac programmers in the audience.

When he said he'd figured out a couple of hard supercomputer problems, members of the audience jumped up excitedly with questions.

After his presentation, a group of nerds followed him to the hotel's bar for drinks, hanging on his every word.

[Mudbug]

The cluster was assembled in less than a month by hundreds of student volunteers who were paid only in soda and pizza for their labor. They ate between 600 and 700 pizzas, Varadarajan estimated

behold the power of cheese. :D

[Sailfish]

Apple Store online employees work on commision, wonder who got the lucky call?

[Stike]

You simply have to smile as you take a look at the ad beside the article:
:D

[wdlove]

LOL! :p

I wonder where the pizza and soda came from? What is deep dish wirh extra cheese?

[Powerbook G5]

I'd work for free pizza and soda! Being a college student, when I am not spending money on my computer or gas for my car, most all of my money goes towards food.

[Rocketman]

With VA Tech pledging to publish their methods and with Apple helping to optimize the system with several people, it seems as if whatever compromises Apple had to make to help VA Tech meet the deadline will be repaid with DOZENS of "supercomputer" projects.

This an astounding development in computing. "Supercomputers for the rest of us" has arrived in practice.

All you need now is an application and a login.

Rocketman

[Rocketman]

Originally posted by Sailfish
Apple Store online employees work on commision, wonder who got the lucky call?

Just look for the RX8 next to all the Camry's.

Mazda RX8 is the Dual G5 of cars BTW.

Rocketman

[sosumi]

"I had to tell them I'd never used the Mac," Varadarajan said. "I'm probably one of the few people who came to the platform by reading the kernel manual."

Thats very funny, I thought he was convinced by some Swither Ad...

[yamabushi]

I pity the poor grad student who attempts to play Chess on this Mac. :)

[sosumi]

I wonder what IBM's reaction is to this whole thing. Apple has more or less accidently moved into their own territory...

[Dippo]

Originally posted by sosumi
I wonder what IBM's reaction is to this whole thing. Apple has more or less accidently moved into their own territory...

Apple does have to buy the chips from IBM, so I don't think that they mind too much. Plus, they can make their own supercomputers with the 970s probably cheaper than Apple.

[sosumi]

Originally posted by Dippo
Apple does have to buy the chips from IBM, so I don't think that they mind too much. Plus, they can make their own supercomputers with the 970s probably cheaper than Apple.

True, but this is a new way of buying super computers; building them yourself from off-the-shelf components. I would expect IBM to prefer not selling just the hardware, but also the expertise at insane margins.

[jayscheuerle]

Originally posted by Rocketman
Mazda RX8 is the Dual G5 of cars BTW.


Similarities:

They both use an engine that no other manufacturer does.

They're both gorgeous.

Difference:

The G5 has TORQUE!! :p

[Doctor Q]

Originally posted by foniks2020
Seriously... do they run SETI during off hours... if ther are any?Supercomputers have their use scheduled ahead of time, by appointment, in time blocks, the way astronomers get appointments for use of the major telescopes. And, of course, if they ran SETI on the cluster, they'd find all the aliens in a single night and spoil the fun for the rest of us! ;)

[tychay]

Originally posted by sosumi
True, but this is a new way of buying super computers; building them yourself from off-the-shelf components. I would expect IBM to prefer not selling just the hardware, but also the expertise at insane margins.

There is a lot more to building a supercomputer than buying them and stringing them together. InfiniBand cost $1.5 million, facilities upgrade cost $2 million (more if you include the original building cost), and installation cost 700 pizzas (assuming you're at a uni that can do the labor). Plus a lot of custom code was written.

IBM will still make money of installation and support. This is nothing but good news for them when you realize that this was inevitable. One of the reasons NCSA beat out PSC and SDSC for funding was because of the move toward commodoty hardware clusters done there--ironically, the VA Tech cluster is built by ex-SDSC people using a PSC-produced "Deja Vu" software... revenge must be sweet.

This really shows how good the 970 is: cheaper than the Itanium2 and twice as powerful as the Opteron. Clock for clock, the I2 is slightly more efficient than the 970--the issue is that that it clocks in 25% slower and costs a hell of a lot more. Methinks Intel will be lowering their price to HP. As for the Opteron, it's a great chip but it doesn't do well for HPC because it's cycle-consuming FPU--it'll still be the buget chip for some other stuff.

As I've said before, I honestly hope that this landmark will cause Linux vendors to seriously consider the 970 for clustering as it deserves more recognition than it has gotten. Such a thing can only help lower Mac prices.

Take care,

terry

[Analog Kid]

Originally posted by Phil Of Mac
They should take all the G5 clusters and connect them together. I will gladly donate my dorm room for this project.

One Cluster to rule them all, One Cluster to find them, One Cluster to bring them all and in the darkness bind them!


This actually isn't a half bad idea for some applications... Figure you've got 2200 nodes at 4 places around the country connected by relatively low bandwidth connections. Most parallelizable problems can be structured hierarchically where you need the fastest interconnect at the leaves and could probably get away with slow communication between clusters.

Think about it-- they're talking run times measured in days or weeks.

Would be a pretty sweet selling point for time on your cluster too: "book ours and other G5 clusters simultaneously and scale your performance."

Could pretty easily get a meta-cluster to beat Earth Simulator, I'd bet.

[Viv]

Originally posted by sushi
Maybe they kept them.

When new Dual 3Ghz versions come out, they could merely swap new Macs for the old ones.

Then sell off the old Macs, with keyboards and mice to offset the cost of the new system.

Afterall, in a couple of years, Dual 2 GHz systems will be slow.

Who knows?! Just an idea.

Sushi

Hey they could laser etch "from the VA Cluster machine number xxxx" on the side or some thing!

Instant collectors item(':D')

Viv

[Telomar]

Originally posted by sushi
Maybe they kept them.

When new Dual 3Ghz versions come out, they could merely swap new Macs for the old ones.

Then sell off the old Macs, with keyboards and mice to offset the cost of the new system.

Afterall, in a couple of years, Dual 2 GHz systems will be slow.

Who knows?! Just an idea.

Sushi I get the feeling you said that as a joke but that's exactly what they did. It's also why every system is fully outfitted in its own right (ie. has an optical drive and graphics card).

[coumerelli]

Originally posted by sushi
Maybe they kept them.

When new Dual 3Ghz versions come out, they could merely swap new Macs for the old ones.

Then sell off the old Macs, with keyboards and mice to offset the cost of the new system.

Afterall, in a couple of years, Dual 2 GHz systems will be slow.

Who knows?! Just an idea.

Sushi

(...because of the previous post...)

But when they buy the NEW 3Ghz, won't THEY come with KBMs too? And probably a little better too - maybe BT?

just another thought. :)