Virginia Tech Supercompuer - Now at #3?

[MacRumors]

MacBidouille links (http://www.hardmac.com/niouzcontenu.php?date=2003-10-28#919) to an interim update on Supercomputer rankings (pdf (http://www.netlib.org/benchmark/performance.pdf)).

The latest results give the Virginia Tech G5 Cluster a score of 9.5 TFlops. Previous preliminary results (http://www.macrumors.com/pages/2003/10/20031021234816.shtml) placed the G5 Cluster at 7.41 TFlops which placed it at 4th place.

The new score of 9.5 TFlops overtakes a Linux 2304 Intel Xeon cluster.

Again, these are still preliminary numbers. Official numbers to be released in November.

[the_mole1314]

This placing has changed, what, 4, times? I mean, we don't we wait another week or so to the final results are in?

[Makosuke]

Sort of like listening to bits of a sports match on the radio--we can't see any of the details, all we get are nonfinal scores every once in a while.

I'm waiting for the ESPN recap, and to find out how well all the other teams scored, for that matter--there could be plenty of new fast clusters around the world (not in Texas, though).

[mvc]

Go Big Mac!!

You bewdy!

[gwuMACaddict]

lets not get grumpy about all the different benchmark numbers...

whats a few teraflops between friends?

:D

[MacsRgr8]

Since Panther this speed increase???

[MrMacMan]

Go SuperComputer...

Though why not wait till the *offical* numbers are put out... I'd rather not like to see Big Mac at 10 instead of #3...

:o

[mymemory]

¿How many Teraflops has my Pismo G3 500? it has 256MB of ram!:rolleyes:

[From Win to Mac]

by the time we get to november, it'll be in first place. Think about it... a TeraFlop or two a week, you can go far with that.
lol

go Bic Mac

[Mr.Hey]

Originally posted by MrMacman
Go SuperComputer...

Though why not wait till the *offical* numbers are put out... I'd rather not like to see Big Mac at 10 instead of #3...

:o

why?, well to build up the suspense of course :rolleyes: :D . MR knows how to milk its cow ;).

[Macmaniac]

#3 would be great, it would show how effective macs can be as supercomputers, and how low cost they are for the performance you get.

[From Win to Mac]

If Apple is smart, they've flooded that VTech Campus with Apple Hardware Engineers straight from California, to try and max that sucker out until there isn't one single processor cycle left to run the dock bounce effect... ;) :D

[manu chao]

The theoretical peak performance of the VT cluster is 17.6 teraflops. In a test with only 128 processors active, they achieved a real performance of about 80% of the theorectical value (80% of what 128 processors could theoretically do). Usually, the more processors you have, the harder it is to get close to the theoretical values.

With 9.5 teraflops, they are now at roughly 54%, which is still rather low, so on can expect some more improvement if everything goes well (and they don't miss the deadline). The biggest Intel (2304 Xeons) cluster achieves 74% of its theoretical value, which corresponds to 7.6 teraflops.

If they could get to 80% they would be at 14 teraflops or second place (there is one at 13.8). But there is no chance that they will become number one, they would have to exceed their theoretical performance.

Another thing that makes the race so interesting, that, although initially their numbers were compared to a list published in June, they are not the only who have started building a supercomputer since then, and other people have been providing new results as well.

And BTW, they have been running a beta version of Panther from the beginning.

[tychay]

Bah, I posted the 9.555 TFlops number on MacRumors (http://forums.macrumors.com/showthread.php?threadid=42624) message board a couple days ago.

Methinks someone at MacBid must have read it.

[Rincewind42]

I knew they could do better than 42% efficiency. The last numbers we saw (around 8.4Tflops iirc) were 48%, and the latest is nearly 54%. 3 more days to eek out a few more optimizations and we may even see 10 Tflops :D .

[Rincewind42]

Originally posted by MrMacman
Go SuperComputer...

Though why not wait till the *offical* numbers are put out... I'd rather not like to see Big Mac at 10 instead of #3...

Well, if they aren't on these preliminary lists yet, then they had better get it together as they only have 3 days left in October...

[macmax]

#2 mostly

http://wired.com/news/mac/0,2125,60821,00.html

[manu chao]

Originally posted by macmax
#2 mostly

http://wired.com/news/mac/0,2125,60821,00.html

As I said before, these numbers were based on a preliminary test running only 128 processors and a comparison of theoretical peak performances.

[iHack]

Imagine that, a panther running that fast on the world's most famous junk food :cool:

M.

(sorry for the lame joke - couldn't resist:))

[aethier]

Well, if we keep up like this, there is a nice chance to see "Big Mac" in 2end place, come November.

aethier

[Mr Jobs]

how many bounce do you think it takes for safari to load on that machine...ha ha ha

[BurntCalc]

Wasn't the deadline Oct. 1? I know they're tweaking the numbers, but the final list submissions have already been sent in, which would place the cluster at between #4 to #6. Or are they allowed to adjust their numbers until Nov. 5?

I'm confused... :confused:

-John

[jettredmont]

Originally posted by manu chao
But there is no chance that they will become number one, they would have to exceed their theoretical performance.



Dude, I know you mean well, but you obviously haven't seen the trendline. We're gaining 2Teraflops every week or two. This time next year we'll be at 100 TFlops or so. Theory, shmeory! #1 is obviously within our grasp!

I mean, jeez, it's like you can't read a simple linear extrapolation of a trend line. And I bet you call yourself an engineer!

:)

[jettredmont]

Originally posted by BurntCalc
Wasn't the deadline Oct. 1? I know they're tweaking the numbers, but the final list submissions have already been sent in, which would place the cluster at between #4 to #6. Or are they allowed to adjust their numbers until Nov. 5?

I'm confused... :confused:

-John

I believe the machine had to be assembled and have preliminary results by Oct 1. The final results deadline is apparently a bit later.

[Phil Of Mac]

Whichever spot it ends up in, you can be sure that Apple's gonna hype it.

But you won't be able to buy your own cluster at the Apple Store!

[theRebel]

Originally posted by Phil Of Mac
Whichever spot it ends up in, you can be sure that Apple's gonna hype it.

But you won't be able to buy your own cluster at the Apple Store!

I am sure that Apple would be happy to sell one to anyone with enough money.

[Phil Of Mac]

Originally posted by theRebel
I am sure that Apple would be happy to sell one to anyone with enough money.

But not at the Apple Store ;)

[Dippo]

Originally posted by theRebel
I am sure that Apple would be happy to sell one to anyone with enough money.

I'll take two please...no wait I found a couple extra million under the cushions, make that three!

[ffakr]

Originally posted by Phil Of Mac
But not at the Apple Store ;)
Well the Apple Edu stores ARE selling 20 packs of eMacs at $599 per unit with the stand.
That's a start. :-)

[settledown]

there is another G5 Mac on the list. Look on page 55 of the 80 page pdf...

It is a 256 processor G5 Mac running at .82 TFlops.

It has the same infrastructure as the "Big Mac"

Who has it?

[alandail]

#1 - cost $350 million - 5120 custom NEC CPUs- 35,860 GFlops
#2 - cost $200 million - 8160 DEC Alpha CPUs - 13,880 GFlops
#3 - cost $5 million - the power macs (2112 CPUs) - 9,555 GFlops
#4 - cost $25 million - Intel Itanium - 8,633 GFlops
#5 - cost $100 million - 4096 Alpha CPUs - 7,727 GFlops
#6 - cost $100 million - 4096 Alpha CPUs - 7,679 GFlops

systems ranked #3, 4, 5, and 6 are all new. The G5 cluster is by far the cheapest and by far the fastest of the new supercomputer clusters. There are a couple of supercomputers in the works that will be much faster than the current #1 - they are both IBM PowerPC based systems.

[Fender2112]

I read down the list of systems and noticed that some of the ones near the bottom had only one or two processors. Has anyone tried running the LINPACk on a single G5 2x2.0. I'm just curious if I might have a top 500 Supercomputer sitting on my desk. Talk about a conversation piece. :D

[Phil Of Mac]

Originally posted by alandail
#1 - cost $350 million - 5120 custom NEC CPUs- 35,860 GFlops
#2 - cost $200 million - 8160 DEC Alpha CPUs - 13,880 GFlops
#3 - cost $5 million - the power macs (2112 CPUs) - 9,555 GFlops


By a pure $ to GFlops conversion, $350 million worth of Power Macs would buy 668,850 GFlops. That would be over 18 times as fast as #1.

[Rower_CPU]

Originally posted by Fender2112
I read down the list of systems and noticed that some of the ones near the bottom had only one or two processors. Has anyone tried running the LINPACk on a single G5 2x2.0. I'm just curious if I might have a top 500 Supercomputer sitting on my desk. Talk about a conversation piece. :D

That's an idea! I hope someone manages to do this.

If we disregard the boost from Infiniband, a dual 2GHz should be somewhere around 9GFlops. It'll probably turn out a little less, but somewhere in there.

[Sun Baked]

There's hope for countries wanting to jumpstart their nuclear weapons program with a few large orders of PowerMac G5s, for not a whole lot of money. :(

[ffakr]

Originally posted by Rower_CPU
That's an idea! I hope someone manages to do this.

If we disregard the boost from Infiniband, a dual 2GHz should be somewhere around 9GFlops. It'll probably turn out a little less, but somewhere in there.

well, a PPC 970 has two double precision floating point units. Altivec doesn't 'do' double precision FP math so they won't help.

Absolute, max, pie in the sky, performance for a dual G5 running double precision FP math would be 2x2x2GHz... = 8GFlops
That would be if it were able to retire two fp instructions every cycle for each processor.
real performance might be around 6MFlops with good code.

guess that would put a Mac in this company...
machine proc. spd. # proc. rMax GFlops
Sun Fire 6800 (900MHz/8MB L2) 4 6.016 28956 1200 7.2
HP Exemplar S-Class SPP-UX 5.2 12 6.005 13320 800 8.6
Cray J920 (10 ns) *** 20 5.917 19456 675 4.0
DEC 8400 5/350 (12 proc 350 MHz) 12 5.904 9548 3010 8.4
SGI POWER CHALLENGE (195 MHz, 2MB cache) 20 5.872 15000 3000 7.8
Sun Ultra HPC 6000 250 MHz (1MB L2 Cache) 14 5.856 15700 960 7.0
DEC Alphaserver 8400 5/440(440MHz, 4MB cache) 10 5.845 9548 1124 8.8
SGI POWER CHALLENGE (195 MHz, 1 MB cache) 22 5.812 15000 2900 8.6
Sun HPC 6500(400MHz 8MB L2 Cache) 8 5.810 39936 1344 6.4

[Phil Of Mac]

Originally posted by Sun Baked
There's hope for countries wanting to jumpstart their nuclear weapons program with a few large orders of PowerMac G5s, for not a whole lot of money. :(

The power to make the infidels pay™

[Telomar]

Originally posted by Phil Of Mac
But you won't be able to buy your own cluster at the Apple Store! Why not? That's where Virginia Tech bought their's.

[Phil Of Mac]

Originally posted by Telomar
Why not? That's where Virginia Tech bought their's.

They just didn't walk into the Apple Store, nor did they just go online and add 1,000 Power Macs to their order. They contacted Apple higher-ups.

[Telomar]

Originally posted by alandail
#1 - cost $350 million - 5120 custom NEC CPUs- 35,860 GFlops
#2 - cost $200 million - 8160 DEC Alpha CPUs - 13,880 GFlops
#3 - cost $5 million - the power macs (2112 CPUs) - 9,555 GFlops
#4 - cost $25 million - Intel Itanium - 8,633 GFlops
#5 - cost $100 million - 4096 Alpha CPUs - 7,727 GFlops
#6 - cost $100 million - 4096 Alpha CPUs - 7,679 GFlops It's worth noting that some of those costs include more infrastructure than was included for the Apple cluster. Not saying it isn't cheap but it isn't an accurate representation of the actual costs to say it was only $5 million.

[ffakr]

Originally posted by Telomar
It's worth noting that some of those costs include more infrastructure than was included for the Apple cluster. Not saying it isn't cheap but it isn't an accurate representation of the actual costs to say it was only $5 million.

you might be thinking of the price of a new Dell cluster that is coming on line. A lot of people got excited to hear the Macs were a lot cheaper, but the Dell cost included construction of a building to house them.
I'm not saying that these figures don't include a lot of infrastracture, but Itanium2s shure aren't cheap. That's about what I'd expect for the machines, cooling, interconnect and racks.

[Phil Of Mac]

Indeed...#1's infrastructure costs included an earthquake-proof bunker.

[Telomar]

Originally posted by Phil Of Mac
They just didn't walk into the Apple Store, nor did they just go online and add 1,000 Power Macs to their order. They contacted Apple higher-ups. They contacted Apple previous to the order to discuss it with them but it was done the same as whenever you do any large order.

[Phil Of Mac]

Originally posted by Telomar
They contacted Apple previous to the order to discuss it with them but it was done the same as whenever you do any large order.

And you know this how?

[dbs900]

Originally posted by ffakr
well, a PPC 970 has two double precision floating point units. Altivec doesn't 'do' double precision FP math so they won't help.

Absolute, max, pie in the sky, performance for a dual G5 running double precision FP math would be 2x2x2GHz... = 8GFlops
That would be if it were able to retire two fp instructions every cycle for each processor.
real performance might be around 6MFlops with good code.



Ummm, note that the analysis above implies that the Virginia Tech cluster has a peak of 8.8TFlops. The Linpack FAQ states that the benchmark must be performed in 64-bit FP arithmetic. It would appear they are only using 32-bit FP in the Altivec. (Note that the Infiniband switch cannot make the FPUs run faster.)

[macphoria]

9.5 TFlops. Whoa.

[Edot]

Originally posted by Telomar
It's worth noting that some of those costs include more infrastructure than was included for the Apple cluster. Not saying it isn't cheap but it isn't an accurate representation of the actual costs to say it was only $5 million.

I am pretty sure that the 5.2 million includes the whole ball of wax. Housing, networking, cooling. The set up was the cost of a few dozen pizzas, so that really doesn't add up to much. I think this is by far the cheapest solution so far. It will be interesting to see if there are any computers that will show up that are not already on the list (besides Big Mac).

[TheDesigner]

Originally posted by ffakr
well, a PPC 970 has two double precision floating point units. Altivec doesn't 'do' double precision FP math so they won't help.

Absolute, max, pie in the sky, performance for a dual G5 running double precision FP math would be 2x2x2GHz... = 8GFlops
That would be if it were able to retire two fp instructions every cycle for each processor.
real performance might be around 6MFlops with good code.


Actually, each dual G5 has two processors. Each processor has two FPUs. Each FPU can do a combined multiply + add per cycle, that is two operations in one instruction. Multiply this by the clock frequency, and you get a theoretical limit of

2proc x 2fpu * 2ops * 2GHz = 16GFlops

And Big Mac has already achieved 80% efficiency when running on only 128 processors, so that would be 12.8 billion double precision floating point operations per dual processor G5 that have already been achieved.

Using Altivec, the numbers would be 2 processor x 1 Altivec FPU x 4 operands in parallel x 2 operation x 2 GHz = 32 billion single precision floating point operations per second theoretical limit.

[Telomar]

Originally posted by Edot
I am pretty sure that the 5.2 million includes the whole ball of wax. Housing, networking, cooling. The housing in terms of the building itself wasn't included and the ongoing operating and support costs weren't included. All are quite considerable. The typical rule of thumb is initial capital outlay is around about 15% of the total cost for its life though.

[aethier]

Originally posted by Sun Baked
There's hope for countries wanting to jumpstart their nuclear weapons program with a few large orders of PowerMac G5s, for not a whole lot of money. :(

Well then, Apple should re-introduce the commercial they had for the original G4, that classified it as a weapon by the US government, and had tanks, etc. ;)

aethier

[gekko513]

Originally posted by Phil Of Mac
But you won't be able to buy your own cluster at the Apple Store!

Sure you can. It's called Xserve. With a Xserve Head Node and some Xserve Cluster Nodes, you'll have yourself a nice little cluster from Apple Store.

It's not G5 though, yet.

[Rincewind42]

Originally posted by settledown
there is another G5 Mac on the list. Look on page 55 of the 80 page pdf...

It is a 256 processor G5 Mac running at .82 TFlops.

It has the same infrastructure as the "Big Mac"

Who has it?

VT - it's a bite of the Big Mac. However, there is an error in the entry - it states 256 processors but a max of "only" 1024 Gflops. 256 970s has a theoretical max of twice that, and it still hasn't been corrected.

[dongmin]

Originally posted by Phil Of Mac
They just didn't walk into the Apple Store, nor did they just go online and add 1,000 Power Macs to their order. They contacted Apple higher-ups.

actually schools do bulk orders all the time. Just go to the apple store and click on the option that says 'buy for your school.'

[dongmin]

direct from the online store (edu):

[alandail]

I believe it was posted somewhere that va tech did indeed order it online - no special discounts beyond the educational price discount, no custom configuration beyond adding ram. The one thing they did get was an assurance they would get their systems in time to meet the top 500 deadline - looks like a brilliant move by apple.

[ClimbingTheLog]

Hey, at $132K/mo, why not?

(actually there are probably some businesses who would be interested in such terms and could afford it)

[dongmin]

Originally posted by ClimbingTheLog
Hey, at $132K/mo, why not?

(actually there are probably some businesses who would be interested in such terms and could afford it)

if you had some spare millions in your back pocket, it wouldn't be a bad idea to build G5 supercomputer clusters for a living. The G5 cluster is coming at 20% (or less) the cost of other comparable supercomputers. You could easily charge people double. Or do something like $1 million per GFlop.

[danbirchall]

Originally posted by ffakr
Absolute, max, pie in the sky, performance for a dual G5 running double precision FP math would be 2x2x2GHz... = 8GFlops
That would be if it were able to retire two fp instructions every cycle for each processor.
real performance might be around 6MFlops with good code.

guess that would put a Mac in this company...
machine proc. spd. # proc. rMax GFlops
Sun Fire 6800 (900MHz/8MB L2) 4 6.016 28956 1200 7.2
HP Exemplar S-Class SPP-UX 5.2 12 6.005 13320 800 8.6
Cray J920 (10 ns) *** 20 5.917 19456 675 4.0

Well, yeah, except there's one tiny thing. Those systems aren't on the Top500 list. Not in November. Not in June. Not since around 1997-1998. The list that starts on page 53 of the preliminary results document (performance.pdf) contains over 1,000 entries. With roughly 40 machines listed per page, anything beyond about page 64-65 (roughly 35-40 GFlops) isn't going to make the cut. :(

Five years ago, a dual-processor G5 would've been in the lower part of the Top500 list. To make it on the list today, you're going to need to cluster at least, say, ten or so. Of course, things will be going even faster come next June, but hypothetically speaking, one could get 100+ Gigaflops on LINPACK from 20-25 dual G5's, so if anyone has a budget of, say, $100K or so and wants to have a "Little Mac" near the bottom of the next list, go for it! ;)

Incidentally, that same document contains (between pages 6 and 46) a lengthy table of LINPACK results from various machines. The original Macintosh apparently was capable of something like 0.38 megaflops, making a dual G5 more powerful than 10,000 original Macs. (The only thing slower than an original Mac on the list was a Palm III PDA. :)

[zzal]

That means there is 1099 ATI Radeon 9600 Pro cards and 1099 superdrives, sitting there, doing absolutely nothing?

[tychay]

Originally posted by Edot
I am pretty sure that the 5.2 million includes the whole ball of wax. Housing, networking, cooling. The set up was the cost of a few dozen pizzas, so that really doesn't add up to much. I think this is by far the cheapest solution so far. It will be interesting to see if there are any computers that will show up that are not already on the list (besides Big Mac).

Not true. The 5.2 million includes the machines and networking, but not the cooling and building infrastructure. The latter two adds $2 million to the price tag for a grand total of $7.2 million. More when you consider most of the housing infrastructure was already built and this was the upgrade cost.

Dell either couldn't meet the ship date or couldn't meet the cost. Not surprising since Dell has little to no experience in HPC and makes crappy servers. (Yes, I've used a lot of Dell servers. They higher performance you go, the weaker the offering relative to HP, Sun, IBM.)

The simple fact is, Virginia Tech tendered all bids from Dell (Itanium), IBM (Athlon and PowerPC), as well as HP (Itanium) with extra time allotted for Dell to meet Apple's price point and ship dates. The Apple machines bid in at less than a half of the cost of the other bids that could meet the Fall Top500 date (i.e. ship by August).

Proof? Infiniband came in at $1.5 million for the cards, routers and cabling (according to the person who runs the VA Tech cluster) which means that the G5 systems cost was around $3.7 million. Every other vendor who could meet the target quoted a price of $8-10 million (again, this is according to VA Tech).

Like I said many times before, this will not hold forever because IBM can design custom systems around the PPC970 starting Q1 2004 which will be cheaper/CPU than G5 towers. I imagine Intel will lower the prices for the Itanium2 in order to compete (or come out with an Itanium3 or speed bump the Itanium2 from its 1.5Ghz). AMD can't compete here because you'd need twice as many Opterons to get a Rpeak in Linpak. That's not to say the Opteron isn't a good chip for other uses (TPC), just not very cost efficient in HPC.

It was a great coup for Apple that involved a planet alignment (Infiniband, the university alliance to create it, and the window before the Fall Top500) that coincided with the release of the G5. By my estimation they didn't even have to lower their systems price at all Wow!

[alandail]

Originally posted by danbirchall
Incidentally, that same document contains (between pages 6 and 46) a lengthy table of LINPACK results from various machines. The original Macintosh apparently was capable of something like 0.38 megaflops, making a dual G5 more powerful than 10,000 original Macs. (The only thing slower than an original Mac on the list was a Palm III PDA. :)

That's funny, because I worked at NASA when the original Mac came out and it actually benchmarked faster than our minicomputer at the time - a VAX 11/750. And there were things we could do on that Mac that we couldn't do on the VAX, yet the Mac cost about the same as the VT-100 terminals we used on the VAX and had a VT-100 emulator.

We bought a ton of those macs once the MacPlus came out and we could add a hard disk. The original Macintosh - it was called simply Macintosh at the time

128k of RAM
400k floppy disk
1 bit black and white display (512x384 resolution if I remember)
mouse
keyboard
dual serial ports
mouse and keyboard port
external floppy port

and that's pretty much it - no hard disk, no ram expansion

cost - $2499

Today you can get the 1.8 GHz G5 machine for $2399.

[Phil Of Mac]

Technically it was called Macintosh 128k. It was followed by the Macintosh 512k and 512KE :)

[alandail]

Originally posted by Phil Of Mac
Technically it was called Macintosh 128k. It was followed by the Macintosh 512k and 512KE :)

I think the Macintosh 128k name only happened after the Macintosh 512k was released and they realized they couldn't call them all exactly the same thing.

[Phil Of Mac]

Why not? It works for the iMac ;)

[ffakr]

Originally posted by alandail
I think the Macintosh 128k name only happened after the Macintosh 512k was released and they realized they couldn't call them all exactly the same thing.
I can check on that. There is an original on the shelf at my old work.. it's the original because it still has the signatures in it (the first run of the mold). That first mold broke so only some of the 128K Macs have signatures in them.

If I recall, just off the top of my head, it does in fact say Macintosh 128K on it.

[ffakr]

Originally posted by danbirchall
Well, yeah, except there's one tiny thing. Those systems aren't on the Top500 list. Not in November. Not in June. Not since around 1997-1998.

I never said the G5 would make the list... just that, by my first incorrect *guess* at where a dual proc G5 would score, this would be the competition.

Just for reference, a quad processor Sun Sunfire with 900MHz processors [from the list I posted earlier] certainly isn't from 1997-1998. More like beginning of this year.
The very latest Sunfire V480 with quad 1.05 GHz processors and 8GB of ram (base config) retails for just under $35,000 yet (based on the fact that I forgot a 2 in my earlier calculations) it shouldn't be able to beat the dual G5 on this benchmark.

It's not fair to compare prices with the 6800 (which is still in production with 1.05 and 1.2 GHz UltSparc IIIi cpus) since the 6800 scales up to 24 processors, and as a result starts at $165,000 for a 4 processor model. :-)

[TheDesigner]

Originally posted by danbirchall
Five years ago, a dual-processor G5 would've been in the lower part of the Top500 list. To make it on the list today, you're going to need to cluster at least, say, ten or so. Of course, things will be going even faster come next June, but hypothetically speaking, one could get 100+ Gigaflops on LINPACK from 20-25 dual G5's, so if anyone has a budget of, say, $100K or so and wants to have a "Little Mac" near the bottom of the next list, go for it! ;)


I would recommend buying 33 dual G5s, for several reasons: First, according to the numbers from VTech (820 GFlops with 64 G5s), you should get 422 GFlops. That would put you at number 241 in the previous Top 500 of June 2003, just ahead of two dozen 192 processor 875 MHz HP Superdomes, so it will take a while before you drop out of the Top 500. Second, you beat all those universities that buy 32 G5s, and there will be _lots_ of those.

33 G5s with school rebate, no superdrive, no modem, cost $82,269. 33 times 4GB of memory at www.crucial.com cost $23,757, and you have 66 256MB chips worth around $3,000 left over. You will need some fast interconnect, that will cost you some money. 33 dual G5s shouldn't produce too much heat, so lets say $150,000 would get you nicely in the middle of the current Top 500 lists and you might stay there for a year or two.