Virginia Tech Cluster Moving to Xserve G5s

[MacEyeDoc]

Makes you wonder...

if VT (and Apple) know that 2.6 or 3.0's won't be in the Xserves anytime soon. Why would you exchange all those boxes for ones of the same speed 4 months after you set it up? Better to wait another 6 months, and get the 3.0's (Steve-o did say summer 2004, didn't he?), and still have time to place #2 in the next supercomputer rankings next fall.

Of course summer on Apple time is more like late September, early October . . .

Maybe ECC memory is a big deal at that level. And they will have lots of extra room to add more Xserves.

Who wants pizza?

 

[Counterfit]

Re: used to make new revision of G5

Originally posted by oliverlubin
dual 2.2 and 2.4 using 90nm G5s and EEC memory for the 2 higher end machines.

We're not going to be seeing ECC (Error Checking Code) RAM in a desktop G5, or any Mac for that matter, Simply because it's slower than non-ECC RAM.

 

[ginoledesma]

REFURB! REFURB! REFURB!... There, I got it off my chest.

Come to think of it, 1100 units isn't too many in light of the 200,000+ units Apple pushed last quarter, but it will probably fill up Refurb stocks for quite some time. And if they DO slap on a "certification" of some sort on the G5 casing (perhaps laser engraving ala iPods), they'd make someone happy. "This unit was part of the legendary Big Mac that made the Top 3 SuperComputers."

If they do a straight 1-to-1 swapping of PowerMac G5s to Xserves, there'd be quite a LOT of vacant (physical) space in their cluster. The dimensions of each PowerMac G5 is, based on specs, 20.1"x8.1"x18.7x" (51.1cm x 20.6cm x 47.5 cm), and each Xserve is about 1.73"x17.6"x28" (4.4 cm x 44.7 cm x 71.1 cm). Now, basing on just the pictures available on the net (since I can't find the dimensions of the racks they used), the Xserves are slightly deeper by about 10" (28" vs. 18.7"), so there'll be about something almost as long as a ruler protudring from the back of the racks. But, using the existing rack setup, they'd be able to stack at most a dozen Xserve in each of the 3-node pack. Assuming each rack contains 4x3 rows, that would be 48 Xserves in each rack compared to the 12 PowerMac G5s in reach rack! That's 4x more power in one rack! And furthering these assumptions, at 1100 PowerMac G5 units, they'd be using about ~92 racks (1100 units / 12 units/rack rounded up). Using Xserves, they'd be using only ~43 of the 92 racks they have. Perhaps they'd be jumping to a 2200-node cluster soon?

Of course, these are all "max" assumptions. They may as well put in PCs there or relocate the racks elsewhere (though that'd mean more vacant space in their cluster), or most likely spread out the Xserves for easier maintenance.

Since they'll be moving to Xserves, each of those G5s are _fully loaded_. I doubt that they'll be keeping all of the parts used, except perhaps for the Mellanox interconnects. Each node has 4GB of RAM (4x1GB) and 160GB of SATA Storage. Selling a PowerMac G5 with that even at refurb is overkill for most, so they might be breaking them down into smaller, more manageable chunks.

Of course, another likely scenario is to sell these cheaper to another institution that wants to build a supercomputing facility. At least they've been guaranteed to work.

 

[vitaboy]

Not only slower, but much, much more expensive. For consumer or even for most workstation applications, there really isn't a need for EEC memory.

 

[ginoledesma]

In addition to a cooling system, perhaps they need a solution to lessen the noise generated by the Xserves? One Xserve alone screams by itself. What more of an orchestra of 1100 screamers?

 

[IndyGopher]

Originally posted by El Tritoma
I'll bet you are an English major with a writing assignment in which you must attempt to use irony and sarcasm. At least I hope you are.

No, if he were an English major, he would have known that plurals don't use apostrophes. (re: "ego's")

 

[rdowns]

Originally posted by Sabbath
I dont think Apple could use these used units to make up ones, surely they would be forced to state them as at least partly refurbished for legal reasons. It would make more sense if they already had some places to take them, ie maybe a few eductaion, government or business orders.

On another hand if we're expecting a powermac update soon, how will these extra rev. A products affect this. I expect it means we will definitely not see a dual low end, as that would lower the price of these products too much. Apple could likely sell them as refurbs at a price above the new base model if its a single 2GHz. I hope it doesnt mean they restrict the low end G5 too much, although common sense would say theres only 1,100 of these so too much importance shouldnt be assigned to them. Especially after the amount of good publicity they have generated for Apple.

One final option give me 1 or 10!

Give me a break. Do you really believe Apple having 1,100 machines to sell as refurbs would affect what speed bumped configurations they would release? Apple is known for head up it's ass marketing but even this is beyond them.

 

[iomar]

That is nice of Apple! I hope Apple is not charging them for it.

 

[isus]

atat did an article about this... they did the math, and i think they said that switching to xserve's would save them a lot of space... the xserve's take up 33% or something?

i can't find that article right now, but...

 

[Macmaniac]

I'd buy a refurb G5 from VTech! Especially if there was engraving! Price em at $2500 and that would be sweet!!!

 

[hexor]

Stats on ECC memory value?

Originally posted by vitaboy
Not only slower, but much, much more expensive. For consumer or even for most workstation applications, there really isn't a need for EEC memory.

I just looked up 1Gb 400Mhz PC 3200 ECC $428.. not too bad.. Are there any stats to show how much more reliable using ECC would be? Maybe it would be worth it...

 

[AidenShaw]

20% more, not "much, much more"

Originally posted by vitaboy
Not only slower, but much, much more expensive. For consumer or even for most workstation applications, there really isn't a need for EEC memory.

Check a site like http://www.crucial.com/store/listmodule.asp?module=DDR+PC3200&Attrib=Package&cat=RAM, you'll see about a 20% extra tax for ECC.

Only $40 per gig to have the peace-of-mind that you won't have random crashes or corruptions....

As to performance, most reports say 1% to 2% performance difference with ECC (see http://www.computer-memory-upgrade-stick.com/ecc-vs-non-ecc.htm for more info).

 

[ZildjianKX]

Setting up the intitial G5 cluster seems like a huge waste of time now. Obviously it didn't have ECC memory before, which surprised me that they would even use DP 2.0 G5s then. Now it all just seems like a big Apple publicity stunt.

 

[legion]

Re: Re: used to make new revision of G5

Originally posted by Counterfit
We're not going to be seeing ECC (Error Checking Code) RAM in a desktop G5, or any Mac for that matter, Simply because it's slower than non-ECC RAM.

close... ECC is Error Correcting Code.

 

[Rex44]

Given the notoriety the Virginia Tech project has received, maybe Apple could sell the old G5s on eBay for 1100 small fortunes?

 

[nagromme]

Not a waste - and who needs to wait for faster chips?

Big Mac 1 was not a waste. VT had two deadlines (one for funding, one for the prestige of the Top 500 list) that were not their own doing. The deadlines are a shame, maybe, but not in VT's control. They had VERY good reasons for finding WHATEVER was best to meet those deadlines. It was PowerMac G5s. If, in the process, they negotiated for something more compact (and ECC) down the road, then that too makes sense. They can judge for themselves if the re-install is worth the benefits or not. Looks like it's worth it.

Think how much bigger the cluster could grow now, in the same installation with the same cooling? 12 PowerMacs in a rack... or 42+ Xserves. That's 2/7 the space!

That means VT will now have room in the same racks for at least 3 and a half Big Macs... probably more.

So... why wait for higher GHz? VT can ADD faster Xserves any time, and keep the old ones too!

 

[El Tritoma]

Waste of Time?

In the real world, unfortunately, very few things have the chance to be done right the first time. This is especially true when you are after funding. The NSF, NASA, and other government funding sources are probably right now making decisions about next year's budget. VT would have lost much more than a few months of work. They would have, effectively, lost about one and a half years if they had waited. The primary goal had to be that they needed to impress some people, and it didn't matter that it wasn't the perfect solution. It was a solution, it was good enough to probably get enough money and attention that they could then continue what they want/need to do. I would be amazed if VT was not thinking about the servers last year. It did not take much imagination to realize that G5s would make it into the servers as soon as possible. Apple, first though, had to do something about the pent-up demand for desktop machines. They could not afford to ignore this. It was not a waste of time. Apple and VT, last summer, had to have formulated a more satisfactory solution than jamming a bunch of desktop machines into a room to build a supercomputer. Just think about that! No reasonable person could have thought they weren't going to redo things. I doubt very much if the VT students think they were wasting their time. They are engineers! They are smart young people. Surely they could figure out that this was only a temporary solution. Besides, wouldn't you just love to be in college at VT and have a chance to put together a world-class supercomputer, and get free pizza and soda!? It would be something to always remember and brag about! It was not a waste of time. Everyone went into it with their eyes wide open knowing what would happen when G5s were put into servers.

 

[ClimbingTheLog]

Originally posted by T.Rex
What a tremendous waste of time. They spend countless days and sleepless nights to get this thing together, spend weeks if not months to finally get it running at top efficiency, just to tear it all down and start again? I would be pretty angry if I were one of the students who volunteered their time to assemble this thing when the administration knew full well it was only going to be around for a few months for the sole purpose of getting them on the supercomputer list this year simply to satisfy their own ego's.

They'll probably have a few work-study students install the new units, a few at a time. I bet two students could easily do 3 an hour, a couple hours a day, over a few months; no need for another army. The cluster can keep on humming all the while.

 

[ClimbingTheLog]

Re: Makes you wonder...

Originally posted by MacEyeDoc
Maybe ECC memory is a big deal at that level.

Yeah, the cluster shoudl be twice as fast for real work. I bet they didn't do it for the benchmarks, but in real usage they need to run jobs twice, to make sure there weren't memory errors.

With ECC you need only run it once. Maybe throw in 1/100 duplicates for good measure, but not every work unit.

 

[i_wolf]

The G5 powermac doesn't need ECC ram. This ram is only required usually for servers or clusters. Usually it carries much higher latency than standard ram meaning that it is slightly slower than standard ram. However in server and cluster environments it provides higher reliablility of data. For workstation apps like Maya, photoshop, Final Cut Pro etc... it would not benefit the G5 in any way at all.. if anything these apps would run slightly slower... because of the added latency incumbant with ECC ram.
However a cluster like virginia tech would benefit because presently part of their cluster is made up of software error checking to make up for the lack of ECC memory. This would slow the current set up down slightly ... i have no idea what difference it woudl make but it would slow it down some what. With the introduction of XServe's with ECC memory they could scrap the software ECC and could potentially gain a speed up since doing a software error check can be compute expensive. Just ot reiterate again though, for the Powermac there is no need to do error checking if anything having ECC memory would hampter its memory latency, it is mainly for server and cluster apps.

 

[Phobophobia]

Originally posted by T.Rex
What a tremendous waste of time. They spend countless days and sleepless nights...

Countless days? Are you still in first grade or something?

 

[woodsey]

I think they should leave in the PM G5s while they install the XServes so they will temporarily have the 2nd fastest supercomputer on earth!

 

[fabsgwu]

Originally posted by woodsey
I think they should leave in the PM G5s while they install the XServes so they will temporarily have the 2nd fastest supercomputer on earth!

Or trade them all in and double or triple the cluster with the space-saving XServes.

But I do agree that it is a publicity stunt or at least smart marketing for Apple--it puts the 4 month old news back at the top... Also, I'm sure the people who volunteered to set up the original G5s wanted to be there--it wasn't a waste of their time (some people in this forum should take a pill).

 

[prb]

ECC

(What a schmuck - I originally posted this to the wrong thread... Sorry)

To me, the most surprising thing about this is that Apple pitch the G5 desktops as viable servers without ECC memory. They won't get far in the corporate server market that way.

I've had the misfortune to administer servers without ECC in the past, and when memory cards start failing it can do some serious damage to the files (and the backups of them!) before you really notice.

As for the VT cluster, ECC is a really big deal... I currently work in an environment with about 200 Sun servers with between 600 and 800G of memory between them (all ECC) and on average we see a few log messages per month where the ECC has corrected a one-bit error and we have maybe half a dozen memory cards per year fail... Usually they degrade before they fail and we pick that up from the ECC corrections and swap 'em.

Assuming Apple non-ECC memory is about as reliable as Sun ECC memory (which seems fair), and scaling that up to the VT cluster of 2200G, they would be seeing a several memory errors every week but with no indication of where they were occuring... Hence the need to run jobs twice and check the results.

Final point: People assume they are going to rip the whole cluster apart and rebuild it. Very unlikely, I'd have thought: They're smart guys and it's a fault tolerant cluster so I'd imagine they'd do a phased replacement, say a cabinet at a time.... Take the G5's in that cab out of the cluster and swap for Xserves, unit test them, then re-introduce them to the cluster. Not much more performance degradation than a single G5 failing in service and having to be replaced.

Pete.

 

[prb]

Originally posted by i_wolf
The G5 powermac doesn't need ECC ram. This ram is only required usually for servers or clusters.

Absolute tosh.

Memory errors happen. A lot of the time they're harmless (who's going to notice 1 bit error in a huge photoshop image, especially if it only occurs once or twice a year?). But when memory starts failing, it can do a lot more damage to your files before you notice if there's no ECC.

The performance overheard of ECC is minimal, so it comes down to the question: Do you want to be sure your computer is giving you the right answers?

I know I do.

Pete.