Virginia Tech Supercomputer?

[MacRumors]

With Dual G5 2.0GHz machines being potentially pushed back a few weeks, ThinkSecret had claimed (http://www.macrumors.com/pages/2003/08/20030829162021.shtml) that Virginia Tech was the likely candidate to receive the first batch of PowerMac G5 2.0GHz machines... after placing a "significant order".

ThinkSecret now (http://www.thinksecret.com/news/virginiatech.html) gives a number to this order-- with report of 1100 units being ordered. The goal of this would be to create a G5 cluster that would become one of the top five fastest supercomputers in the world.

These machines are expected to ship by next week.

[King Cobra]

-->In related news, Virginia Tech University reportedly placed a significant order of maxed-out dual-2GHz G5s, supposedly to form a large cluster boasting remarkable speed and storage. Apple reportedly bumped this order to the front of the line, in front of first-day orders, to get them out the door all at once. Representatives for Virginia Tech agreed to look into the report but didn't respond to follow-up requests for comment.


I remember hearing/reading about the first G4 supercluster about two years ago, and I found that incredible. Imagine the technology + clusters we'll have in about a year.

Dual 3.xGHz clusters on 980 chips sound unremarkably impressive now, but in two years it will seems just plain "fast".

[BigJayhawk]

Let's see . . .

1100 G5's x 2 Processors = 2200 Processors

2200 Processors x 2 Ghz EACH = 4400 Ghz

As I recall, this would equal 4.4 TeraHertz!!!


Uh, how many Frames Per Second could you get on Quake with that? How about rendering the entire Finding Nemo II Movie in about ten seconds???

OK, WOW!

[FelixDerKater]

Mmmm. Budget cuts... Raise my tuition and cut more classes, but buy one of the fastest computing clusters in the world... Finances make no sense around here.

[TheFish]

man, all i have to say is "wow"

[Freg3000]

This is really strange, but I am incredibly overjoyed about this story.

I have no knowledge of Virginia Tech's history with Macs, but this is amazing. TS says that if they get it all up in running so it can qualify for the Linpack Supercomputer list, it could be in the top 5.

I am sorry that this is screwing up people's orders, but I think this is worth it (it might not be if I had a G5 on order....). This is really cool. :)

[asim]

holy sh... um...

does anyone know how efficient is os-x in clustering vs other operating systems? for instance, if one unit (2 cpus) performs at x gigaflops, what percentage of 10x would 10 units perform? and is the limiting factor here going to be the os, how well the software is written, or will it be the communication between the units (gigabit ethernet? fw800?)?

so they are getting 2200 2ghz g5 processors... i could encode aac files pretty quickly with that kind of power

afc

[york2600]

Who said they were using OS X? They could just load Linux on it and use it just for the 970s, but then again they could have just bought them from IBM then. Who knows.

[tex210]

My local news kept showing these pc clusters and all I could think was... what a waste of money... if only they had bought G5's.

[zon7]

Remember that the G5 fans are Software controled and that no Linux dist now (not Yellow Dog not other) support that feature.
If the cluster is true they must be using OS X

[Dahl]

Originally posted by Freg3000
I am sorry that this is screwing up people's orders, but I think this is worth it (it might not be if I had a G5 on order....). This is really cool. :)

Wow! It's the G 5500. :)

I agree with you, from a marketing point of view, it may be a good move by Apple to ship to them first, it may be a good news story here and there and the bragging rights are nice too. I hope the people waiting for their own G 5's will understand why Apple shipped to Virginia Tech first and feel that's it's good for the Apple platform and their users in the end.

[Alexander]

Originally posted by FelixDerKater
Mmmm. Budget cuts... Raise my tuition and cut more classes, but buy one of the fastest computing clusters in the world... Finances make no sense around here.

I wouldn't be surprised if they had a donation that was specifically to be used for this purpose. Happens all the time. I agree that it's kind of backwards, but remember the golden rule: he who has the gold makes the rules.

I think this is pretty damn spiffy, though. Nice PR, for sure. I wonder if Apple is making a profit on these 1100, and if so, how much.

If was was just some profs thinking they need 4GHz to run Word (well...maybe they do; it's Word ;) ), I'd be pissed, but I don't mind my order being delayed a little bit for some pro-Apple cause. (Ok, it helps that this particular machine's going to be running Panther, and the later the build the better...)

asim, the limiting factor is almost always the algorithm that's being used. Massively parallel computing is a whole different ballgame; you can't just fire up iTunes and get 2200x the encoding speed, you have to run specialized programs that know how to split their work into 2200 separate processes on different machines. (Think SETI@Home, Folding, etc.) Some algorithms also rely heavily on the bandwidth used to connect machines, though all but the craziest should be fine with Gigabit Ethernet.

Yay! G5s are now in the same class as PS2s! ;)

[ImAlwaysRight]

So why in the world can't VT wait a month to receive their 1100 computers and instead the masses who ordered first receive their computers first? What is the rush? Like one month will make a difference to them. I'm sure it makes more of a difference to the 1100 or so individuals who are waiting on their PowerMacs because of this order. And you can't tell me VT placed their huge order the day these G5's were announced. It takes more than a few hours to receive approval for a purchase of that size. :mad:

ADDITION: What will win more people to Mac platform, actually SEEING friends, professionals, etc. using PowerMac G5's, or hearing news about 1100 G5's being used in a cluster? How many people are actually going to cluster 1100 computers? Probably none.

So to me it is better for Apple to get G5's in the hands of everyday people and make the cluster wait their turn. Also, Apple can still benefit from ANNOUNCING that VT has a huge order which would be fulfilled later in Sept. for a supercluster. So I don't understand why this is such a great move on Apple's part.

I would imagine this order was placed late July when Apple changed their shipping time from "August" to "7-10 weeks". As I said before, a purchase this size would not have been decided overnight.

[Mudbug]

So do we have any information on what said supercomputer would actually be used for?

I bet it could fold pretty well... :D

[Somebody]

Originally posted by zon7
Remember that the G5 fans are Software controled and that no Linux dist now (not Yellow Dog not other) support that feature.
If the cluster is true they must be using OS X


Why? They may just be setting the fans to run all the time. Sure, it would waste some power (but not much as a percentage of what the boxes are drawing anyways).


Further, how do you know that the software to control the fans is actually part of OS/X? Maybe it's some small, cheap embedded controller on the MB that handles it, in which case the OS doesn't need to think about it at all?

[shadowfax]

Originally posted by FelixDerKater
Mmmm. Budget cuts... Raise my tuition and cut more classes, but buy one of the fastest computing clusters in the world... Finances make no sense around here. it has to be worth it though, if they let you just go into the room where they all are once. what surprises me is why they didn't wait for xServes. those would look considerably cooler together, take up less space, and so on. another perplexing thing is, if apple can get such corporate orders, why aren't they focusing more on systems like the xServe?

[shadowfax]

Originally posted by Somebody
Why? They may just be setting the fans to run all the time. Sure, it would waste some power (but not much as a percentage of what the boxes are drawing anyways).


Further, how do you know that the software to control the fans is actually part of OS/X? Maybe it's some small, cheap embedded controller on the MB that handles it, in which case the OS doesn't need to think about it at all? no, it's software controlled. that's why the G5s are running on 10.2.7 (partly). just like the light sensors on the 17 inch powerbook.

you comment about setting them to run all the time is funny too. how do you do that in linux, if you can't control them?

[etoiles]

Originally posted by BigJayhawk

How about rendering the entire Finding Nemo II Movie in about ten seconds???



The renderfarm used to render Finding Nemo had over 2000 processors (Sun UltraSparc of some sort), and it took an average of 8 hours to render a single frame (not sure how many frames were being rendered at the same time, though).

just to put computing power into perspective ;)

btw, Pixar's new renderfarm has 1024 2.8Ghz Xeons...

[AidenShaw]

ThinkSecret said that the rush was to get this cluster into the fall "Top 500" list.

Since the "Top 500" is ranked by *measured* LINPACKD performance, we'll finally see some GFLOP measurements for a Power Mac that can be compared to other machines.

This will be much better than the GFLOP statements made by Apple marketing folks, those are ridiculous theoretical instantaneous peak numbers.

(And, by the way, AltiVec cannot be used on LINPACKD....)

[asim]

Originally posted by ImAlwaysRight
...And you can't tell me VT placed their huge order the day these G5's were announced. It takes more than a few hours to receive approval for a purchase of that size. :mad:


a project this size is usually a part of a huge government or foundation grant that is planned out long ahead of time. they may have also been working with apple behind the scenes and have made their commitment PRIOR to the g5 announcement. just as it wouldn't come about in a day or so, you can't get a group to invest millions (1100 x ~5000 = 5.5million) in a new technology even in the two months since it has been announced.

this must have been in the works before june.

afc

[vouder17]

Well i went to applestore and did a order of 1100 Powermac dual 2ghz G5's, just for the fun of it to see the price..........

It came to
$9 651 400.00


This is considering the powermac has 8GB memory, 2x250 GB hard drives and ATI 9800 pro video cards

I did not include any discounts but i am pretty sure that apple must have given them some kind of discount or something that would lower that price!!!

[AidenShaw]

Originally posted by vouder17

It came to $9 651 400.00


Did you calculate the cost of wirung up a 2500 amp electrical service?

How about the air conditioning for this 300,000 watt heater?

Is the 1100 port GigaBit Ethernet switch included?

Or 1100 InfiniBand cards (about $2M) and the 1100 port InfiniBand fabric (out of this world)?

And the 1100 Cat5e or Cat6 network cables (probably $25K or so). InfiniBand cables, power cables, ....

So, $10M for the computers is probably less than half the cost (and they probably don't need the 2x250GB disks - they need the CPUs).

[trog]

Originally posted by vouder17
Well i went to applestore and did a order of 1100 Powermac dual 2ghz G5's, just for the fun of it to see the price..........

Wow! And the estimated shipping was tomorrow! ;)

[etoiles]

Originally posted by vouder17

It came to
$9 651 400.00


how many airmiles do you get if you put this on your AmericanAirlines credit card ? A free trip in Steve's private jet ?

[trog]

If you were to charge that to Discover Card you would get approximately $97,000 cash back. Not bad. :)

[Macmaniac]

Like I said in the other thread is it to late to apply to Virginia Tech???;)

[sachmo]

How long do you think it would take Word to launch on this cluster?

No seriously, VT needs it right away to fix Michael Vick's leg, genetic engineering!:p

[michaelperez1]

Ever heard of 50 cent's G - Unit well this is Virginia tech's G5 -Unit lol

[SiliconAddict]

Originally posted by shadowfax
you comment about setting them to run all the time is funny too. how do you do that in linux, if you can't control them?

The hardware is fault tolerant is why. If the service that is controlling those fans up and fails, your system would overheat. In the case of the G5 and many Unix systems (I know the sys where I work has this feature.) if the software stops controlling the fans the hardware simply turns them all on. In the case of loading Linux it would be no diff then if the service on OSX failed. All 9 fans would be running.

PS- Give it 3 months after the G5 starts shipping in earnest. You'll have software out there to control these fans. I’m quite sure there are plenty of *nix fans out there chomping at the bit to get their hands on a G5 system. Did I say 3 months? Make that 3 weeks.

[gothamac]

Originally posted by etoiles
The renderfarm used to render Finding Nemo had over 2000 processors (Sun UltraSparc of some sort), and it took an average of 8 hours to render a single frame (not sure how many frames were being rendered at the same time, though).

just to put computing power into perspective ;)

btw, Pixar's new renderfarm has 1024 2.8Ghz Xeons...

But not the next one. Not if you believe the Pixar spokesman in the G5 video.

[shadowfax]

Originally posted by SiliconAddict
PS- Give it 3 months after the G5 starts shipping in earnest. You'll have software out there to control these fans. I’m quite sure there are plenty of *nix fans out there chomping at the bit to get their hands on a G5 system. Did I say 3 months? Make that 3 weeks. i TOTALLY agree. it WILL happen for sure. in fact, if VA tech is spending 10 million on these babes, they just may use a *nix distribution and have someone write the software to control the fans via cooperation with apple or just reverse engineering. it's certainly probably not hard to do. *nix folks have written their own drivers for years.

[mainstreetmark]

Why did Virginia Tech need 1100 all at once? Couldn't they spread this out a litte bit? Deliver them over the course of two months? Where they gonna even keep 1100 boxes until they get around to installing them?

[jcgerm]

I actually go to VT and I'm not sure where the money came from for all this. We're still hurting from budget cuts.

But as for this being a rumor...my roommate was talking about a meeting at the Corporate Research Center (CRC) this morning at 10 AM where they needed people to help build a big supercomputer. This morning was an info session apparently.

[shadowfax]

Originally posted by mainstreetmark
Why did Virginia Tech need 1100 all at once? Couldn't they spread this out a litte bit? Deliver them over the course of two months? Where they gonna even keep 1100 boxes until they get around to installing them? if you spent $10 million on a bunch of computers, would you want to just put them in storage? no. they are more than likely going to install them and connect them as quickly as possible so they can get them up and running.

[Vlade]

Originally posted by vouder17
Well i went to applestore and did a order of 1100 Powermac dual 2ghz G5's, just for the fun of it to see the price..........

It came to
$9 651 400.00


This is considering the powermac has 8GB memory, 2x250 GB hard drives and ATI 9800 pro video cards

I did not include any discounts but i am pretty sure that apple must have given them some kind of discount or something that would lower that price!!!

I have a feeling that they won't need 1100 Radeon 9800's and that their smart enough to NEVER buy apple memory, and 550 Terabytes is overkill.... they probably only need 10 or so :p

[wiljo]

Originally posted by asim
holy sh... um...

does anyone know how efficient is os-x in clustering vs other operating systems?
[...snip...]
afc
I seem to recall a govt sponsored research project in distributed OS technology at CMU back in the '80s. It was called Mach. You know, basic abstraction of memory, process and communication in a micro-kernel, on which one could host an os personality such as BSD. Oh yeah, it was led by this guy named Avi Tevanian. Who moved this tech to a company named NeXT, who moved to a company named Apple...

Cheers

[shadowfax]

Originally posted by wiljo
I seem to recall a govt sponsored research project in distributed OS technology at CMU back in the '80s. It was called Mach. You know, basic abstraction of memory, process and communication in a micro-kernel, on which one could host an os personality such as BSD. Oh yeah, it was led by this guy named Avi Tevanian. Who moved this tech to a company named NeXT, who moved to a company named Apple...

Cheers hahaha! very nice tongue in cheek. kudos. thanks for the info.

[testnull]

They are going to be using OS X.

They will also be using MPICH, though it would be better it they used MPI/Pro.

[arn]

For people asking why they need them early/first.

read the linked article.

arn

[blogo]

Now why didnt they wait for g5 xserves? :rolleyes:

[Somebody]

Originally posted by shadowfax
you comment about setting them to run all the time is funny too. how do you do that in linux, if you can't control them? [/B]


Presumably Apple has the fans set to revert to some sort of reasonable default behavior when the OS fails to provide any sort of control. And the only reasonable default in that situation is for them to simply run full-time. While obviously OS X is the primary system envisioned for these machines, it's not as if the hardware engineers designing them were unaware that some people would want to run Linux and other OS's on them.


Aside from that: If VT made providing a small Linux driver capable of controlling the fans one of the requirements for Apple to land this contract, I'm sure that Apple quite happily provided the miniscule development effort required for said driver.


To go back to the original point: While it's quite possible that the nodes in this supercomputer will run OS/X, it seems unlikely. Much more effort has been put into clustering Linux than OS/X over the past few years, and the primary advantages that OS/X provides over Linux (a reasonable user interface, and a fantastic API for building GUI apps) are irrelevant in a supercomputing application.


To be fair, I can think of a much stronger reason for the cluster to run OS X is marketing: I'm sure Apple will want to be able to brag about this thing in their marketing literature, and they can brag much more loudly if it's running OS X. So I can easily imagine that they've cut a deal with VT where they do the legwork of getting the clustering infrastructure working on OS X, in return for the resulting marketing capital. Controlling the fans? A completely insignificant issue, in comparison.

[shadowfax]

Originally posted by Somebody
Presumably Apple has the fans set to revert to some sort of reasonable default behavior when the OS fails to provide any sort of control. And the only reasonable default in that situation is for them to simply run full-time. While obviously OS X is the primary system envisioned for these machines, it's not as if the hardware engineers designing them were unaware that some people would want to run Linux and other OS's on them.now now, it's presumable. or maybe not. the apple in know would just make it not work without OS X, by default. they have never seemed to have any inclination about supporting other OSes than their own. hell, they don't even support their own 9 anymore, really. you sure can't make that run on this computer.

and as you said, and has been said by others before you (myself included), it's not hard to make these drivers that will control it. apple more than likely wouldn't do it, if you ask me. but there are others who can and will. just like linux on the iPod.To go back to the original point: While it's quite possible that the nodes in this supercomputer will run OS/X, it seems unlikely. Much more effort has been put into clustering Linux than OS/X over the past few years, and the primary advantages that OS/X provides over Linux (a reasonable user interface, and a fantastic API for building GUI apps) are irrelevant in a supercomputing application. Originally posted by wiljo
I seem to recall a govt sponsored research project in distributed OS technology at CMU back in the '80s. It was called Mach. You know, basic abstraction of memory, process and communication in a micro-kernel, on which one could host an os personality such as BSD. Oh yeah, it was led by this guy named Avi Tevanian. Who moved this tech to a company named NeXT, who moved to a company named Apple...don't forget to read threads you post on ;)

[MrMacMan]

Originally posted by ImAlwaysRight
So why in the world can't VT wait a month to receive their 1100 computers and instead the masses who ordered first receive their computers first? What is the rush? Like one month will make a difference to them. I'm sure it makes more of a difference to the 1100 or so individuals who are waiting on their PowerMacs because of this order. And you can't tell me VT placed their huge order the day these G5's were announced. It takes more than a few hours to receive approval for a purchase of that size. :mad:

ADDITION: What will win more people to Mac platform, actually SEEING friends, professionals, etc. using PowerMac G5's, or hearing news about 1100 G5's being used in a cluster? How many people are actually going to cluster 1100 computers? Probably none.

So to me it is better for Apple to get G5's in the hands of everyday people and make the cluster wait their turn. Also, Apple can still benefit from ANNOUNCING that VT has a huge order which would be fulfilled later in Sept. for a supercluster. So I don't understand why this is such a great move on Apple's part.

I would imagine this order was placed late July when Apple changed their shipping time from "August" to "7-10 weeks". As I said before, a purchase this size would not have been decided overnight.


I think your wrong 'IAmAlwaysRight' people will hear and see how powerfull the G5 Can be.

There is always demand for new products. Being slightly later or earlier will not mean much.

If they are your friends they will come back and see your new copmuter.

Mudbug -- That would fold insanily fast, turn in a new unit what, once every .2 seconds?

:eek:

Originally posted by AidenShaw


(And, by the way, AltiVec cannot be used on LINPACKD....)

Is SSE or Hyperthreading able to run?

It seems unfair if they aren't off.

Anyway they want to to be in the record books.

If they don't set it up fast, and right, they need time to repair fix... ect.
And as for the $9 Million ... They don't need All those 9700 And they said they would be upgraded to either 4 Or 8 GB not All 8 GB.

[shadowfax]

Originally posted by ImAlwaysRight
So why in the world can't VT wait a month to receive their 1100 computers and instead the masses who ordered first receive their computers first? What is the rush? Like one month will make a difference to them. I'm sure it makes more of a difference to the 1100 or so individuals who are waiting on their PowerMacs because of this order. And you can't tell me VT placed their huge order the day these G5's were announced. It takes more than a few hours to receive approval for a purchase of that size. :mad: maybe what apple considers more valuable than making everyone wait their turn is showing people with real cash the respect it demands.... wouldn't you like to get $10 million for you computers? i mean, suppose VA tech came to them and said, look, we can pay for all this, but we need it ASAP, before x date, otherwise we will have to find another vendor. do you put them in line behind everyone else, or give them VIP treatment? you give them VIP treatment. in the long run, people will be just as impressed with their boxes 3 weeks from now as they would have been today. i am sure it's causing some large amount of trouble, but it's the way business works.

[rt_brained]

Boy, I bet they have to make a pretty strong building to house all those G5s.

Did you see what just one G5 did to that guy's house in the commercial?

[andyduncan]

Originally posted by shadowfax
in the long run, people will be just as impressed with their boxes 3 weeks from now as they would have been today.

3 weeks delay from this? I don't even think it's THAT bad. We're talking about 1100 boxes here, Apple was said to have over 100,000 pre-orders. Sure, not all of those were 2ghz machines, but 1100 is still a drop in the bucket.

[pilotgi]

Wouldn't they use OS X Server to run a cluster?

[mchendricks]

I can say that VT has had a relationship using Macs for years. I remember them REQUIRING computer science majors to have their own Mac II in 1988. We were running Apple's implementation of Unix.

Go Hokies!

[whooleytoo]

Originally posted by shadowfax
now now, it's presumable. or maybe not. the apple in know would just make it not work without OS X, by default. [/B]

Apple's docs state if OSX isn't booted, the other OS must take control of thermal management.

They don't say what happens if the OS doesn't take control - do the fans go 100% or do they just shut off. But, since the fans (presumeably) start long before the OS has booted up, it's likely the OS's role is to wind down the fans when possible, it's not responsible for keeping them running.

p.s. 9900 fans? Can anything be done to harness all that wind energy?? :-)

[dongmin]

Originally posted by Vlade
I have a feeling that they won't need 1100 Radeon 9800's and that their smart enough to NEVER buy apple memory, and 550 Terabytes is overkill.... they probably only need 10 or so :p

if you buy through the edu store, a basic dual 2ghz G5 (minus modem and superdrive), runs you $2493. Times 1100 is $2,742,300. A 1gb stick of ram runs you about $350; 8 sticks would be $2800; times 1100 is $3,080,000. So about $5.8 mil for the cpus.

[cc bcc]

Wouldn't it be better for them to buy 1100 xServes G5? :)

[legion]

It would be a good idea to run G5 Xserves... if they existed As far as we know, there will be no G5s in Xserves, ever

About the fans... I'm guessing those fans are not meant to run continously for long periods of time. Can you imagine the failure rate??? I'd hate to have to be the one to replace those. There will have to be software built to control those.

As for delivery, they could space those out. If anyone believes you can set up a cluster like this overnight, you're a sucker. Just laying down the building framework will take a week or two. Then when they do get the machines, each system will have to be tested on-site before they are attached to the cluster (the chances of all 1100 being flawless would be very small) This project will take awhile. Between Apples' (notorious) delay(s) in deliveries and the time it'll take to set this up (especially since you're dealing with both revision A hardware and a first-time situation), I don't think Virgina Tech is going to make it for the Fall Super 500.

I'm guessing the real rush here is that AMD has recently had a lot of major contracts to drop new clusters into the top 10 of the Super 500, which could knock Apple's cluster out of the top 10 before it even gets a chance. That would be a huge loss for Apple and it's marketing department (especially as a first attempt into the 500 which could spur more orders) I'm guessing IBM is also helping out since they have much more experience in these super-structured computers. Otherwise, we would have heard of a huge hiring spree in Apple's support center to manage such a cluster (as it stands now, it'd take the whole lot of them to support just the 1100 G5s at VT)

[asim]

if they decide to replace some of the units in the cluster with quad-4ghz machines in two years, they can take some towers out and use them as desktop machines in computer labs or elsewhere.

this was a big selling point when the oil exploration company hess decided to build a linux cluster from desktop machines. they were renting a unix supercomputer from ibm for over a million a year, and they decided they could use the same software (perhaps after a minor recompile) and get equivalent results for less than a million dollars one-time cost (they would own the machines).

or they could just keep adding new machines without decomissioning the older ones.

afc

[pilotgi]

Wouldn't they use OS X Server to run a cluster?
Anybody?

[dguisinger]

Originally posted by pilotgi

Anybody? [/B]

Because OS X has never been designed for large scale clustering, nor would they want to handle all those CPU cycles of running a GUI on every box (talk about a waste of billions of cycles a second). If you want to get in the top 5 with speed records, you have to go with a headless OS with no graphical subsystem.

[asim]

Originally posted by dguisinger
Because OS X has never been designed for large scale clustering, nor would they want to handle all those CPU cycles of running a GUI on every box (talk about a waste of billions of cycles a second). If you want to get in the top 5 with speed records, you have to go with a headless OS with no graphical subsystem.

how bout darwin?

[pilotgi]

Because OS X has never been designed for large scale clustering, nor would they want to handle all those CPU cycles of running a GUI on every box
My question really was isn't OS X Server a better OS for clustering than OS X.

Doesn't Apple's server software have the option of not using the GUI?

[applekid]

How do you make a Mac cluster anyhow? I wouldn't mind combining 2 G4 towers at my dad's office and use it to make a fast Quake machine. Again with my naive thinking (probably) :o

Anyways, just think, one of those G5s could've been yours :rolleyes:

[zaphoyd]

You can quite easily run mac os x sans gui...


clustering would not work with quake, you would need to use software specially designed to distribute its processing over a network. Given that most mac games cant even distribute themselves over two processors millimeters away from each other sharing a bus and system controller i would find it very unlikely to have a cluster-able game.

[3-22]

A friend who is into Mac's and a Virginia Tech grad called me about this prior to it hitting the web. I thought it seemed a bit odd, I guess not...

Sounds like apple screwed some customers over this... Sounds like Apple just wants bragging rights on being a super computer so bumped other orders for VT. Not that I blame them it will be good marketing if it makes it into the top 5... Still sucks for those who get bumped... I heard talk of $150 for bumped orders, that would soften the blow.

[sedarby]

Originally posted by applekid
How do you make a Mac cluster anyhow? I wouldn't mind combining 2 G4 towers at my dad's office and use it to make a fast Quake machine. Again with my naive thinking (probably) :o

Anyways, just think, one of those G5s could've been yours :rolleyes:

Sure kid, just fan the fires. Apple cannot afford to alienate one of their key markets so yes putting 1100 on the truck to VT is a very smart move.

OTOH, I wonder how many people cancelled their dual 2 ghz G5 orders? Do you think it was over 1100?

[mak_suzu1]

Originally posted by 3-22
A friend who is into Mac's and a Virginia Tech grad called me about this prior to it hitting the web. I thought it seemed a bit odd, I guess not...

Sounds like apple screwed some customers over this... Sounds like Apple just wants bragging rights on being a super computer so bumped other orders for VT. Not that I blame them it will be good marketing if it makes it into the top 5... Still sucks for those who get bumped... I heard talk of $150 for bumped orders, that would soften the blow.

So,
IBM released beta compilers with open MPI which is necessary to the cluster.
AND
Virginia Tech got the machine during 1st week of September.

Now we could see within 5 among TOP500, newspaper and TV will say, "Virginia Tech today anounced ...

[Phil Of Mac]

Originally posted by whooley
p.s. 9900 fans? Can anything be done to harness all that wind energy?? :-)

Most supercomputers have dedicated air conditioning units. They might just deactivate the fans utterly. This is a "special order" after all...

Originally posted by legion
It would be a good idea to run G5 Xserves... if they existed As far as we know, there will be no G5s in Xserves, ever

It's not unreasonable to assume that there will be, eventually.

Originally posted by legion
As for delivery, they could space those out. If anyone believes you can set up a cluster like this overnight, you're a sucker. Just laying down the building framework will take a week or two. Then when they do get the machines, each system will have to be tested on-site before they are attached to the cluster (the chances of all 1100 being flawless would be very small) This project will take awhile. Between Apples' (notorious) delay(s) in deliveries and the time it'll take to set this up (especially since you're dealing with both revision A hardware and a first-time situation), I don't think Virgina Tech is going to make it for the Fall Super 500.

Virginia Tech, just like any other university, has ready access to a large part-time workforce that doesn't have to be paid much. I suspect a lot of this work isn't that difficult :)

[scem0]

And they are all maxed out too! wow.....

I should call and beg for one. I'm quite a good actor, I think i could really stir up their compassion and they would end up giving me one. Hmmmmm which role should I play..... Single dad with 3 young kids and not enough time? Nah.... 17 year old boy who is living on his own because his parents kicked him out for his love of computers and now he has to work 5 jobs to scrape by? Nah.... 16 year old boy whose parents refuse to help him out with buying a new computer, so he is stuck with a PC, and is too much of a tight-wad to buy the computer with his own money? Wait...... that wouldn't be a role at all ;).

scem0

[Powerbook G5]

I can't even imagine Folding on that thing...MacRumors could be #1 in like 20 minutes after that thing is done...

[Frobozz]

I have to laugh every time someone ranks a story like this negative.

Yeah, Apple having one of the top 5 fastest super computers in the world is bad _how_, exactly? Is it because (currently) 10 guys aren't getting their machines for a month? Hmmm? :-)

If anyone was shocked their order was delayed for any number of reasons then they must be new to the Apple scene. I feel your pain, guys, but that's the mode of operation at Apple.

[tychay]

Originally posted by legion
I'm guessing the real rush here is that AMD has recently had a lot of major contracts to drop new clusters into the top 10 of the Super 500, which could knock Apple's cluster out of the top 10 before it even gets a chance. That would be a huge loss for Apple and it's marketing department (especially as a first attempt into the 500 which could spur more orders) I'm guessing IBM is also helping out since they have much more experience in these super-structured computers. Otherwise, we would have heard of a huge hiring spree in Apple's support center to manage such a cluster (as it stands now, it'd take the whole lot of them to support just the 1100 G5s at VT)

There are some caveats that need to be placed here:

IBM is the vendor on contract to build most of these AMD Opteron clusters (http://www.infoworld.com/article/03/06/24/HNibmcluster_1.html) so it is never a case of Apple/IBM v. AMD or some such. Right now, IBM is the only tier-one vendor of AMD CPUs, so nearly all high scale HPC clusters will go through IBM. (Yes, you can build a renderfarm through RackSaver (http://www.racksaver.com/), but this isn't the same thing.) Besides, AMD rents IBM Fishkill for design and fabrication.

To my knowledge there are only three Opteron clusters in the works that would be eligible for the Top500: Japans AIST 2116CPU @ 11 Teraflops (http://www.businesswire.com/cgi-bin/f_headline.cgi?bw.073003/232115458), China's Dawning Information 2000 CPU @ 10 teraflops (http://www.dawningusa.com/about-us.html), and Red Storm 10,000CPU @ 36Tflops (http://news.com.com/2100-1001-962787.html?tag=nl). If such a Mac cluster were to be built, it would beat out the second and be completed before the third: this is hardly able to push such a computer from 3rd out of the top 10!

I agree that it doesn't seem practical to have a cluster up and running in time for the Fall Top500 (http://www.top500.org/list/2003/06/). Usually clusters take a while to set up. This leads me to think that the 2200 G5 cluster of VATech only slightly related to the 2x2Ghz delays.

Actually Mac OS X makes a better base OS to cluster than Linux. Yes, more research has been done clustering Linux boxes (http://www.linuxvirtualserver.org/), but Mach is definitely superior in this regard. Besides, I'd imagine they wouldn't focus on handling clustering at the kernel level, but would run an application level message passing clustering system like MPich. I have no idea if LinPack can be run on top of this layer since I don't do HPC. Macs have been clustered on small scales for a long time this way.

As for waiting for XServes, I wonder what everyone is smoking. Clearly the thing to wait for is the 4-way or 2-way PPC970 IBM Blades and rackservers (1Q 2004) with AIX and the #1 vendor in the top 500 than a G5 Xserve with an uncertain release date and a vendor with no previous clustering experience.

Finally, in the PC world, the fan is controlled by the BIOS, not the OS. This is from experience, because I set machines to always on full fan when doing memory tests--in one case, my computer kept shutting down because the fan would keep turning off, thank god it was a Pentium.

I don't believe this is the reason for the delays, but it does allow one to be amusing. As one of my friends put it, ``Damn them, Virginia Tech probably has my machine.''

[Frobozz]

Originally posted by 3-22
Sounds like apple screwed some customers over this... Sounds like Apple just wants bragging rights on being a super computer so bumped other orders for VT. Not that I blame them it will be good marketing if it makes it into the top 5... Still sucks for those who get bumped...

Yeah, I don't condone it, either. But, if anyone really thought their personal order for a G5 dual 2 GHz to play UT2003 on was going to get precedence over the 5th faster super computer in the world, they're smokin' herb.

[zach]

Originally posted by Powerbook G5
I can't even imagine Folding on that thing...MacRumors could be #1 in like 20 minutes after that thing is done...

we wouldn't just be number 1, they would run out of new units to give us!

:D

[Lord Bodak]

Originally posted by shadowfax
now now, it's presumable. or maybe not. the apple in know would just make it not work without OS X, by default. they have never seemed to have any inclination about supporting other OSes than their own. hell, they don't even support their own 9 anymore, really. you sure can't make that run on this computer.
[/B]

I certainly hope they wouldn't do that! Lots of people leave their machines on 24/7 nowadays... what if the OS crashes? The fans just stop and the machine overheats? Scary thought.

[MrMacMan]

Originally posted by sedarby

OTOH, I wonder how many people cancelled their dual 2 ghz G5 orders? Do you think it was over 1100?

Hell no.

If anyone doesn't want one, screw you and good job.

I think anyone who ever wanted a computer should want one of the fastest computers.

Gimme a break, there are no mass cancelations.

No way in hell.

Originally posted by Powerbook G5
I can't even imagine Folding on that thing...MacRumors could be #1 in like 20 minutes after that thing is done...
Yup, turn in a WU every .2 Seconds.

:eek: :eek:

/\- Plug for MacRumors Folding Team -/\ (http://forums.macrumors.com/showthread.php?threadid=21908)

[inkswamp]

Originally posted by zon7
Remember that the G5 fans are Software controled and that no Linux dist now (not Yellow Dog not other) support that feature.

I'd like to hear more about this. I don't doubt that the fans would be software controlled, but is that software specifically part of the operating system? I don't have to think for long to come up with scenarios where that would be potentially disastrous (i.e., system freeze--rare but can happen; putting the machine to sleep when it needs the fans to keep running; running another operating system, as you said.) It seems to me that the control for such a thing would come from the ROM or from firmware. It just seems to be a bad design to make the funtionality of the fans dependent on the operating system itself. If I understand these things, control for hardware components is usually a part of the function of firmware. I would guess that it would be perfectly fine to run Linux on G5s. In fact, given that Apple probably knows that there is a segment of their customers who do specifically that with their hardware, why would they exclude those potential sales?

[freemidnight]

This is the kind of subject that the "big mouth encounter" comes in!:D :D :D

[NavyIntel007]

Originally posted by sachmo
How long do you think it would take Word to launch on this cluster?

No seriously, VT needs it right away to fix Michael Vick's leg, genetic engineering!:p

Actually, they need it to figure out how to beat Miami... :D :cool:

[dguisinger]

Originally posted by inkswamp
I'd like to hear more about this. I don't doubt that the fans would be software controlled, but is that software specifically part of the operating system? I don't have to think for long to come up with scenarios where that would be potentially disastrous (i.e., system freeze--rare but can happen; putting the machine to sleep when it needs the fans to keep running; running another operating system, as you said.) It seems to me that the control for such a thing would come from the ROM or from firmware. It just seems to be a bad design to make the funtionality of the fans dependent on the operating system itself. If I understand these things, control for hardware components is usually a part of the function of firmware. I would guess that it would be perfectly fine to run Linux on G5s. In fact, given that Apple probably knows that there is a segment of their customers who do specifically that with their hardware, why would they exclude those potential sales?

Lets not forget the YellowDog folks have an Apple OEM agreement in place for repackaging the G5s.....I'm sure its a microcontroller in the chipset that has its own ROM, and manages it on its own... the only place the OS comes in is fine-tuning it, by letting the microcontroller know that hey, the OS thinks different, slow down the fans.

[sedarby]

Originally posted by MrMacman
Hell no.

If anyone doesn't want one, screw you and good job.

I think anyone who ever wanted a computer should want one of the fastest computers.

Gimme a break, there are no mass cancelations.

No way in hell.


Yup, turn in a WU every .2 Seconds.

:eek: :eek:

/\- Plug for MacRumors Folding Team -/\ (http://forums.macrumors.com/showthread.php?threadid=21908)

Careful mate the fates will get you for that. Actually, it was a joke. Wouldn't it be ironic if exactly 1100 orders were cancelled due to the month delay? I would hope it would be less, much less. Oh well, carry on then and good day to you.;)

[Mac Kiwi]

Pixar ordered hundreds of G5s themselves,so if Virginia tech ordered 1100,and say Pixar ordered 2000 or something theres our hold up for delivery,not to mention other big orders we dont even know about.



What was it over 100,000 G5 - 2ghz machines,I wonder what percentage go to big organisations and universitys etc.




Heres a giggle for you guys.A university close to where I live has always been a Mac unversity.Recently a percentage of the students bitched and moaned because they couldnt be bothered learning another OS,so the Uni ditched hundreds of machines and bought PCs,but then a couple of viruses hit and they have been fixing the mess ever since.One of the law lecturers who is a friend of mine is a Mac fan and has been walking around campus with a grin as she made them leave her a Mac {still a handful of Macs there} and is getting everything done as per usual,whilst everyone else screams and rants as things get fixed :D

[Rocketman]

Originally posted by cc bcc
Wouldn't it be better for them to buy 1100 xServes G5? :)

Nope. This way the students can use them as standalone workstations and all unused CPU cycles can be hogged by Pooch and similar programs for cued processing tasks. Not unlike folding.

X-serves are "dedicated" servers.

I was thinking that if VT ordered a mere 1100 (out of the 100k+) and impacted delivery dates to some detectable extent, what would happen if Pixar broke down and bought some G5's to abandon those crappy Xeon CPU's? People on this site would whine loudly and persistently.

And I would be amused no end.

Not to mention if the VT cluster works out, people (institutions) might actually duplicate that and buy G5's 1000 at a time. Imagine the shock and awe!

Merely Rocketman

http://www.v-serv.com/-upload/avatar2.jpg

[mclosers]

I go to VA Tech and would rather see them cut the teaching program and buy a super computer any day of the week. :) Anyways we already have a building called the math emporium that houses around 490 flat screen iMacs for math students. To see the warehouse/room is jsut amazing before they upgraded to those they had about 490 G4s Here is a quicktime vr of the g4s http://www.emporium.vt.edu/emporium/VisitorsWebsite/VisitorsMain.html
I'm looking forward to finding out where this cluster is being put and taking pictures/video for you all :D I hope Tech finds some aliens cures breast cancers or folds some dna with this thing.
From my calculations this should reach around 15.4 Teraflops
14 Gigaflops per duel g5 times 1100. Take some away for the clustering and this is still going to be aroudn the 3rd fastest machine in the world. 1st being built using 10000 processors, so it's a bit unfair.

[pjhornak]

vah tek

they need the cluster to hack the BCS computer and keep them in the running for a bowl bid...

[haobaba1]

it has to be worth it though, if they let you just go into the room where they all are once
**************************************************

I plan on being there the set it up!

[DrGonzo]

why would you buy 1100 first revision g5s? I understand they want the machines ASAP... but still to spend at least $2mil on a bunch of machines that are brand spanking new with no small scale usage and certainly no large scale, and also were, for the first time, mass produced... just doesn't seem like a great way to do things. Then again, maybe this cluster isn't that important.

[CybrCyfr]

After reading up on YellowDog linux and their Blank Lab cluster management software, VT might have this planned out pretty well afterall.

But then again I have heard talk of clustering for OSX on large scales. Apple did produce a clustered XServe right?

It will be interesting to see what OS VT chooses in the end. Anyone out there have first hand knowlegde of the clustered backplane to be used? I would love to see the whitepaper on this this.

[visor]

Originally posted by vouder17
Well i went to applestore and did a order of 1100 Powermac dual 2ghz G5's, just for the fun of it to see the price..........

It came to
$9 651 400.00



Of course - the superdrives and Graphic cards and HD's are completely useless in those, You also won't need modems and the like.
You can probably save a thousand bucks just leaving unneccessary stuff away.

btw. Why use Powermacs in a cluster anyway? this kind of thing belongs in a server rack. It's extremely unefficiant to use desktops as a super computer.

But then - hopefully we'll be seeing xservers G5 soon.

[visor]

Originally posted by CybrCyfr
After reading up on YellowDog linux and their Blank Lab cluster management software, VT might have this planned out pretty well afterall.

But then again I have heard talk of clustering for OSX on large scales. Apple did produce a clustered XServe right?


I guess you can still buy them. cluster nodes - at the same price as single processor xseres - but with two g4's

[Trimix]

Originally posted by asim
a project this size is usually a part of a huge government or foundation grant that is planned out long ahead of time. they may have also been working with apple behind the scenes and have made their commitment PRIOR to the g5 announcement. just as it wouldn't come about in a day or so, you can't get a group to invest millions (1100 x ~5000 = 5.5million) in a new technology even in the two months since it has been announced.

this must have been in the works before june.

afc

If what you say has any merit to it and I do not doubt it one minute then it is incredibly poor planning on Apple's side because the 1100 units were a given way before the announcement and they should have prepared better.
If this cluster news is now being used for an excuse then that is especially p...poor performance on the side of apple.
Who cares if they use 1100 units ? What about the 1100 consumers waiting ? The small businesses, the artsy crowd who need this for their next customer account ? No, if this is really true, then Apple have either their priorities wrong or their planning is extremely amateurish.

[danhaupt]

Check out the slashdot thread on this topic:

http://apple.slashdot.org/apple/03/08/30/1838222.shtml?tid=107&tid=137&tid=187

[danhaupt]

From the /. thread: http://apple.slashdot.org/apple/03/08/30/1838222.shtml?tid=107&tid=137&tid=187

In my email the other day, I received this letter:
Hello all,

This email is to serve as invitation and notice of impending Terascale Facility assembly assistance. For those receiving this info for the first time know that Virginia Tech is building a top 10 supercomputer from scratch and we need your assistance. We do have one stipulation to volunteerism and that is you must not be a wage employee of the university. Grad students on GTA/GRA are fine as well as others outside the university that may wish to volunteer.

We are expecting to receive machines next week!!! Yikes! In preparation for the assembly process, we need to get volunteers together at the AISB (Andrews Information Systems Building), 1700 Pratt Dr., this weekend. We are planning to have a process orientation session start at 10:00 AM on Saturday, August 30, and last no longer than an hour. We can give you a few more details about the project if you show up and have not been before.

There are many things that need to be covered and many new volunteers needed. We have posted an electronic sign-up sheet for proposed shifts at (link deleted) We will need folks to sign-up as either a primary volunteer or on-call/backup person that we can call and bring in if we are short people. We know this is a very busy time for everyone and we want to get this done and over with quickly so it will not affect other work that needs to be done across campus. Once we have a definite date for the deliveries we will send out notification to those folks that said they were available on that day. We will have 48 hours notice of shipment, 72 hours notice of delivery. The machines will arrive on a staggered, every other day, schedule. Three shipments are expected for the total number of machines.

Orientation today was postponed, however, so I won't have more details until Wednesday =/ I'm looking forward to helping out, though.

[danhaupt]

From the /. thread: http://apple.slashdot.org/apple/03/08/30/1838222.shtml?tid=107&tid=137&tid=187

For those unbelievers, here's a little proof [vt.edu] and maybe a bit more [vt.edu]

Here's an official word [vt.edu] (search for Teraflop).

Also, here's the original e-mail that went out (a month ago) They never mentioned Apple though:

> Date: Mon, 28 Jul 2003 17:36:46 -0400
> From: Jason Lockhart <multimedia@vt.edu>
> Subject: Terascale Assembly Assistance
>
> Hello all,
>
> As you may know the College of Engineering in conjunction with the
> university Information Systems and Computing organization are
> building
> a 10 TeraOp, 1100 node supercomputing cluster. We are in need of
> volunteers to assist in three areas of assembly; cabling, RAM and PCI
> card install, and machine racking. We would like to get as many
> volunteers as possible.
>
> Some logistical things, there is not adequate parking for every
> volunteer to drive and park at the Computing Center. We
> would ask that
> volunteers either carpool (4 or more to a vehicle) or take the BT.
> Initial cabling will be done with the power cables beginning this
> coming Friday, August 1st. We have yet to set a start time,
> but I want
> to get an idea of who is available to assist on that day as well as
> availability for the weekend and early next week. I will forward the
> time to arrive as soon as I nail it down...should be tomorrow. The
> power cabling represents one third of the overall cabling that will
> need to be done.
>
> I need to get a list of those who will be assisting to the facilities
> people at the ISB as soon as possible. This list will be
> used to allow
> volunteers access to the building and the machine room. If
> you're not
> on the list you will not be allowed in! We want to have at
> least 30 to
> 40 people working at any one time on the project. If you know others
> that will be interested in assisting, please have them email me so I
> can get them on the roster.
>
> Thank you for your willingness to participate, and please respond and
> have others respond with "Terascale" in the subject so I can
> filter the
> volunteers properly.
>
> Thanks.
(E-mail signature removed)

[danhaupt]

From the /. thread: http://apple.slashdot.org/apple/03/08/30/1838222.shtml?tid=107&tid=137&tid=187

For the ones who are questioning this existence, the order is shipping, the racks (a ton of them) are there in the main Computing Center server room. First they required all servers to be moved innto racks. Then they started moving servers around, including removing the Petaplex [vt.edu]. The power has been upgraded in the server room (the UPS backup generator actually). This caused a morning of basically all the important servers on campus having to go down for one day in the summer - I hated waking up to go switch off machines for that one. The AC has been upgraded to accomidate the huge amount of heat to be put out. It was't until I heard about the cluster that all the chages in the Machine Room made sense. Now they're recruiting help to do the grunt work of putting all the machines in the racks.

The stated objective was to be on the next 500 list. Dell and HP were considered, but they couldn't fill the order in time (possibly as they have made announcements of other large clusters recently) and Apple promised delivery after someone leaked the story of the cluster meetign with Dell and HP to Apple and Apple jumped at the chance.

Basically, the story is not a rumor from the point of view of the geeks on campus who have been effected by the preperations. I'll probably post the /. link to the campus geek list (If someone hasn't beaten me to it).

I'm disapointed about this being only on the Apple section of /. since a cluster this size is noteworthy of the frontpage. (Rumor - and this is rumor sice I haven't goe to direct sources on this - is that it will not be running OS X, and probably BlackLab or YellowDog [terrasoftsolutions.com] or SuSE.)

[peterjhill]

Originally posted by FelixDerKater
Mmmm. Budget cuts... Raise my tuition and cut more classes, but buy one of the fastest computing clusters in the world... Finances make no sense around here.

The University will most likely be doing more than run word or quake on these machines. They will be doing serious research. Govt. agencies will be paying professors to have projects to do all sorts of research. The Uni will make plenty of money from this investment. I doubt that any student tuition money went to this project.

[dietsoda]

looks like YDL is perfectly at home with the G5

Terrasoft Store - G5's! (http://www.terrasoftsolutions.com/store/)

Or at least will be...

* We are working hard to provide Yellow Dog Linux support for the Apple G5s. In the mean time, we are pleased to offer Apple G5s with a partition for both Yellow Dog Linux and OS X. Your G5 will ship with the current set of CDs, 'Getting Started' book, and sticker. While your computer will immediately provide OS X, we will deliver the updated install CDs and 60-days Install Support --free

[brsoar]

Dan is exactly right. This has been going on for some time now.
I've seen the room where the cluster is going to go. They have already cleared out the space and have all the racks assembled. I've been told the G5's will have 4g ram per each system, and they are going to use a distro of linux. They are expecting all the sysytems in a week or two.

[Tiauguinho]

This is so cool! Finally I can piss off my PC friend's about Apple being on the Top 5 of the Most Powerfull, Fastest, Badest Kick-Ass computers in the World! This is very good for Apple, will give them credit for the PPC they now use. Apple PR can now brag a lot and investors will feel confident. Way to go Apple!

[bommai]

Originally posted by shadowfax
no, it's software controlled. that's why the G5s are running on 10.2.7 (partly). just like the light sensors on the 17 inch powerbook.

you comment about setting them to run all the time is funny too. how do you do that in linux, if you can't control them?

Apple has specifically stated that if software is not controlling the fans, they run all the time. This is a common fail safe mechanism in any system.

[AidenShaw]

Originally posted by Tiauguinho
This is so cool! Finally I can piss off my PC friend's about Apple being on the Top 5 of the Most Powerfull, Fastest, Badest Kick-Ass computers in the World!

Perhaps you didn't look at the link - #3 is currently a Xeon cluster, and there's another Xeon cluster and an Itanium cluster in the top 10.

http://www.top500.org/list/2003/06/?page


On the summary page - 100 of the top 500 are Intel Pentium systems, and 22 more are Itanium and AMD.

I'm sure that you'll be pissed if your PC friends toss those figures back at you! ;)

And remember, the rankings are by measured sustained LINPACK double-precion performance, not by marketing GFLOPs. The G5 should do well (lots of memory bandwidth, dual float units) even though AltiVec cannot be used.

But until the machine is built and the numbers are run - "top 5" is a finger in the wind guess. :rolleyes:

[iDONTsteal]

ANY increase in the shipping computers from Apple helps Apple... increases profit (EVEN if these 1100 are a wash... the scale of production will yield some added economy).

Perhaps it will actually speed the release of the next G5's or speed the price reduction of the G5's shipping right now!

either is GREAT (did I mention I'm happy it wasn't a story about VT getting a cluster of DULLs?

:p

[OutThere]

Thats really sweet....a mac supercluster...it ought to give apple some publicity.

[alandail]

Originally posted by dguisinger
Because OS X has never been designed for large scale clustering, nor would they want to handle all those CPU cycles of running a GUI on every box (talk about a waste of billions of cycles a second). If you want to get in the top 5 with speed records, you have to go with a headless OS with no graphical subsystem.

This is flawed - there are no CPU cycles being used by an idle GUI.

[alandail]

also, 1100 machines isn't going to cause much of a delay on over 100k orders - it might bump everyone back a few hours.

[jettredmont]

Originally posted by FelixDerKater
Mmmm. Budget cuts... Raise my tuition and cut more classes, but buy one of the fastest computing clusters in the world... Finances make no sense around here.

Umm, the funding is most likely not from "Virginia Tech". It is most likely a grant for this specific task.

[jettredmont]

Originally posted by shadowfax
no, it's software controlled. that's why the G5s are running on 10.2.7 (partly). just like the light sensors on the 17 inch powerbook.

you comment about setting them to run all the time is funny too. how do you do that in linux, if you can't control them?

The speed is throttled down by the OS (OS X only). The default speed for the fans is full-bore. This was stated shortly after the announcement.

[The Grimace]

Originally posted by ImAlwaysRight
As I said before, a purchase this size would not have been decided overnight.

I read on the slashdot thread (http://apple.slashdot.org/article.pl?sid=03/08/30/1838222&mode=nested&tid=107&tid=137&tid=187) regarding this that VT has been in the market for a cluster for a while, and actually looked into Dell and HP before going G5. So it seems as if the purchase was approved before the G5 was announced.

(tig)

[guzhogi]

If you think the G5s are going to be fast now, just wait until Apple releases a truly 64-bit version of OSX, saying that VT does use OS X. That plus 64-bit apps, the G5s would be smoking!

As for the $9 Mil+ VT paid for them, don't forget that VT is an EDUCATIONAL INSTITUTION! That means that they'd probably get a discount. They might even have gotten still bigger discount considering they bought 1100!

[iDONTsteal]

I sorta doubt that the user experience / GUI will go 64-bit for quite some time.

Doing so would require simultaneous maintenance of OS X 32 & 64-bit builds... this will undoubtedly happen eventually, but I imagine Apple will make a managed effort at limiting the duration that this simultaneous maintenance would have to be endured... perhaps once G5's are in the majority of new sales? .. or when some major clients request such a thing??


of course... Apple has already stated that 64-bit apps will function now... just that the 'Finder' will remain 32-bit... since it's just an app like others you'd run (Graphic Converter, AOHell, Safari, Seti@Home, your Folding selection... and so-on)... I don't think it matters much.

I still want one NOW NOW NOW!! Time will tell:D

[AidenShaw]

Originally posted by iDONTsteal
Apple has already stated that 64-bit apps will function now...

Do you have any reference that says that Apple claims that 64-bit applications are supported (that is, applications that see a 64-bit virtual memory space and deal with 64-bit memory pointers).

I didn't think so.... 10.2.7 (G5) and 10.3 support 32-bit applications, which can be optionally compiled to use a 64-bit integer data type with 32-bit pointers.

That's not a 64-bit application. For proof - AltiVec has been manipulating 128-bit chunks of data, but nobody calls the G4 a 128-bit system. G3/G4 and Pentium chips have 64-bit floating point, but nobody calls them 64-bit computers.

It's the width of the address pointer that matters, not the width of the largest datum.

[iDONTsteal]

DOPE... you are indeed correct


it'll be a nice upgrade though... once supported fully.. right? :-P

[guzhogi]

It would be nice if the G5 & later processors were completely 128-bit, like Altivec & Apple shipped a completely 128-bit OS for it. Saying that everything else hardware-wise (memory paths, system controllers, etc.) were all 128-bit (or whatever the highest one is if it's higher) and operated at the same clock speed, these computers would be ****** fast! Damn! I'm a Computer Science major @ school & we learned about the basic I/O controllers & how the processors work (though still @ a very rudimentary level), if all the hardware ran @ the same bit-rate & clock speed, there'd be little t0 no bottleknecks when different parts of the hardware are communicating. However, there's not very much out there (for consumers anyway) in software to fully take advantage of this. But when there is, that would be a great day. Please correct me if any of my info. is incorrect.

[PHGN]

Originally posted by vouder17
Well i went to applestore and did a order of 1100 Powermac dual 2ghz G5's, just for the fun of it to see the price..........

It came to
$9 651 400.00


This is considering the powermac has 8GB memory, 2x250 GB hard drives and ATI 9800 pro video cards

I did not include any discounts but i am pretty sure that apple must have given them some kind of discount or something that would lower that price!!!


Why get top spec ones for a cluster -- as i understand it what they’d need is the processors and some RAM -- plus the mobo and a small HDD to make 'em work. Oh, and a network card; fibrechannel or gigabit. No graphics card, no optical drive, no big hard disk, etc.

And at 1100 units you bet they got a discount. And an advertising requirement in the contract.

[AhmedFaisal]

Completely dumb. The G5 Xserve Upgrade is only 2 months away (max) and price wise the Xserve Cluster node is less expensive than a dual CPU client. That way they would have had easily stackable 1U machines that were designed for exactly that purpose which would save both space and money. Again a sign that government folks same as university people are not capable to reasonably use money.
Cheers,

Ahmed

[AidenShaw]

Originally posted by PHGN
and a network card; fibrechannel or gigabit

They'll be using InfiniBand ... (up to 10 Gbps with low overhead and latency)

[AidenShaw]

Originally posted by guzhogi
It would be nice if the G5 & later processors were completely 128-bit.... I'm a Computer Science major @ school & we learned about the basic I/O controllers & how the processors work

Keep studying, and you'll realize how silly your statement is.

The "bitness" of a processor almost always refers to the size of an address pointer register. The PPC970 (which Apple puts in the Power Mac G5) has 64-bit addressing, but only supports 42-bits of physical memory. That's 4096 GiB of RAM, though, which is unlikely to ever be reached with this chip.

The G4/P4/PPC970 already have 64-bit, 128-bit, and 256-bit internal datapaths - which is what's important for performance.

A "64-bit" computer lets each process address more than 4 GiB of RAM. Useful, but consider that IBM thinks that 42-bits is way more than will be needed during the lifetime of this processor.

And of course, Apple is shipping the PPC970 with a 32-bit operating system (yes, Panther is also 32-bit) - so the 4 GiB limit per process still exists.

Your wish for 128-bit won't be needed while any of us are alive - but I'm sure that the starships' Enterprise computers are more than 64-bit. It's silly.

[Vlade]

Originally posted by guzhogi
It would be nice if the G5 & later processors were completely 128-bit, like Altivec & Apple shipped a completely 128-bit OS for it. Saying that everything else hardware-wise (memory paths, system controllers, etc.) were all 128-bit (or whatever the highest one is if it's higher) and operated at the same clock speed, these computers would be ****** fast!

Actually, 128 bit Altivec and a 128 bit CPU are VERY different from what I understand. Altivec processes 4 separate 32 bit chunks at once (so basically 4 instructions in one clock cycle), while a 128 bit processor would just process one 128 bit piece of data, thus being VERY VERY Accurate, but not any faster.

And 128 bit addressing is useless, because right now we use 32 bits, which is 4 GBs or RAM, and the G5's use 42 bit addressing which is 4096 gigs of RAM, 128 bit addressing would let you have over

100,000,000,000,000,000,000,000,000,000,000,000,000

Bytes of RAM.

[guzhogi]

Regular consumers probably wouldn't need that much RAM. Maybe people doing something like the Earth simulator might, but still. What I was trying to get @ was that data would get to each part of hardware @ the same time (MHz) & could carry the same amount of data (bit-size). But I don't claim to be an expert on computers. Sure, I'm a CS major, but the stuff I learned so far is still REALLY rudimentary. So in you're replies, stay nice. We're all friends here.

[Phil Of Mac]

Originally posted by guzhogi
Regular consumers probably wouldn't need that much RAM.

Yeah, 640K is enough for anyone :)

[zedster]

Can you imagine the sound of 1100 startup chimes!
The neighbors will know all about it when they reboot.
Z.

[Phil Of Mac]

Also, if they waited for the Xserves, well, the Xserves wouldn't necessarily have the same megahertz rating.

So there.

[AidenShaw]

Originally posted by guzhogi
What I was trying to get @ was that data would get to each part of hardware @ the same time (MHz) & could carry the same amount of data (bit-size).

That's a good idea - and it's what even 32-bit computers are doing today with wide (64/128/256 bit) datapaths.

Having addresses wider than needed, however, is a liability. Many people think that recompiling a 32-bit program with 64-bit pointer will make it faster. More than likely, however, it will be a little bit slower - those pointers are bigger, and take up more memory, more memory bandwidth, and more of the caches.

If you don't need more than 4 GiB of memory in your application, you're better off with 32-bit addressing.

[guzhogi]

I think those who say that xServe clusters would be better are right. I don't think that, when clustering, you need so many hard drives, video cards, etc. Besides, xServes are a lot smaller and can be put in a rack. With a regular desktop like a Power Mac, there's the stylized case, all the PCI card space, CD/DVD & hard drive bays, etc. For clustering, all you really need is a hard drive for each motherboard, a networking device like ethernet or fiber channel for each motherboard, only one CD/DVD drive, preferably a burner for backup, and a case just big enough for all that. And since xServes can be put into racks, the room(s) that they'd be stored in would be more organized. However, I don't know how much heat a G5 & appropriate motherboard would give out so it may have to be put into a 2U chassis, but still, that's pretty good. And w/ an xRAID, I heard that Maxtor is shipping 320GB Serial-ATA drives so there's a lot of room. Also, think of the power requirements! W/ all those Power Macs, the switches/hubs used to connect them, etc., that gets kinda expensive.

[sacrilicious]

Originally posted by Phil Of Mac
Also, if they waited for the Xserves, well, the Xserves wouldn't necessarily have the same megahertz rating.

So there.

And I'm sure if they're ordering that many machines that Apple would've told them that the XServe would be out shortly, if it was a concern.

[Phil Of Mac]

They're already mounting the G5 towers in customized racks. It would save real estate if they used Xserves, sure, but which of the two exists right now?

[alandail]

two good reasons not to go with XServes for this project

1 - the G5 XServe is not out yet
2 - when these machines are replaced, they can be resold to students and/or professors as desktop machines.

[cr2sh]

So this one time right, I'm on spring break in virginia beach.. and I meet a chick from virginia tech.. and end up nailing her. So I guess in a way, now she's screwing me? damn that karma! :)

[rainman::|:|]

Originally posted by trog
If you were to charge that to Discover Card you would get approximately $97,000 cash back. Not bad. :)

if you have a credit card with a ten million dollar limit... i want to go have a few drinks with you...

:)
pnw

[firewood]

It's the width of the address pointer that matters, not the width of the largest datum.

The "bitness" of a processor almost always refers to the size of an address pointer register.

Not always true. The Apple II is usually considered an 8-bit personal computer; but the 6502 CPU uses a 16-bit PC, address bus, and address pointers (zero page indirect). The Nintendo N64 usually configured the MIPS R4300i CPU in 32-bit address, 64-bit data mode.

[AhmedFaisal]

Originally posted by alandail
two good reasons not to go with XServes for this project

1 - the G5 XServe is not out yet
2 - when these machines are replaced, they can be resold to students and/or professors as desktop machines.

Regarding 1.:

The will be out within a reasonable timeframe.

Regarding 2.:

Get real. Before a university/government institution is allowed to sell any kind of hardware to anyone some years go by or the things die so badly its not worth repairing the thing. The newest Macs my university for example is allowed to sell at the moment are 7xxx series.

As for use anywhere else. You can bet that these machines will be stripped off anything not useful in a cluster node (unless they wen't totally bonkers when they placed the order). So they will have either a Rage 128 Graphics Card or none, same with optical drive, none or CD-ROM, I would even bet none for both and they do network installs and an absolute minimal HD (since a cluster node doesn't need much HD space). So if they want to ever use them outside the cluster they would have to upgrade them with new Graphics Card, Optical Drive and HD.
Cheers,

Ahmed

[shadowfax]

Originally posted by AhmedFaisal
Regarding 1.:

The will be out within a reasonable timeframe. that's what they said about powerbooks like 3 months ago.

[Phil Of Mac]

null

[dguisinger]

Originally posted by shadowfax
i think he probably knew this, and you are just being a semantic jerk. he was saying that they are just selling those series of macs. do you know the plural of series? it's series. it's also a generic term that can be used to refer to a line of products... i don't see why you fried him like that, other than to try to impress us with your knowledge of gimpy apple products from x years ago. that'll be a neiner. i knew the point he was making, and you did too--that his university can only sell really old apple computers. so please, don't correct spelling and minor word confusions.

lol, that was great

[Phil Of Mac]

null

[groovebuster]

Originally posted by alandail
also, 1100 machines isn't going to cause much of a delay on over 100k orders - it might bump everyone back a few hours.

Exactly what I thought! So everybody who is pissed about the delay caused by an order 1100 G5s shouldn't have slept in school when the teacher explained multiplication... This is less than the througput of one day, so what's the big deal?

groovebuster

[groovebuster]

Originally posted by Phil Of Mac
I don't know what your problem is, Shadowfax, but jumping to conclusions and flaming me without knowing better is no better than what you accused me of.

You are so wrong... you started it and he just put it into perspective. For me it also was unnecessary how you corrected that series issue. And to claim that it was for the "uninformed folks" who are new to the platform is a bit weak. If they are new to the platform, they probably couldn't care less about the details you tried to brag with, because it is very unlikely that they will ever be in the situation to work with a machine like this.

Oh... and BTW... your explanation was wrong anyway... The so called 7xxx cases were also sold in November 1997 as the first generation of G3s... Ooops! You don't have to thank me for correcting you. I just thought you didn't know this... you must be new to the platform! ;)

groovebuster

[Phil Of Mac]

null

[Phil Of Mac]

null

[Wombatronic]

Originally posted by wiljo
I seem to recall a govt sponsored research project in distributed OS technology at CMU back in the '80s. It was called Mach. You know, basic abstraction of memory, process and communication in a micro-kernel, on which one could host an os personality such as BSD. Oh yeah, it was led by this guy named Avi Tevanian. Who moved this tech to a company named NeXT, who moved to a company named Apple...

Cheers

Hrmm...

Alternately, it was lead by Rick Rashid, who now heads up MS Research. He appears as principal author on most of the papers, anyhow... Somehow I don't think I will convince any of you. :) (nor that Mach was about extensibility rather than distribution)

[cr2sh]

How about if we retreat from personal attacks on our fellow members and turn our animosity back to where it rightfully belongs.. at those goddamn hokies! Where do they get off? ;)

[Phil Of Mac]

Originally posted by shadowfax
i am not trying to make a flame war. i jumped on you, quite harshly, for correcting someone for something minute and pathetic. you proceeded to correct him--with a correction that wasn't really even valid--and lecture us about the details of some macs that were not mentioned in the interest of detail but rather example. i think this is rude. i am not alone in thinking such. to defend your actions, you turned the accusations back upon me multiple times, and fell behind the "oh, i was just trying to enlighten everyone" guise which is quite common among people who fail at attempting to do what you did.

All right. I've deleted the posts in question, and I'm sorry.

[groovebuster]

Originally posted by cr2sh
How about if we retreat from personal attacks on our fellow members and turn our animosity back to where it rightfully belongs.. at those goddamn hokies! Where do they get off? ;)

You are right... sorry! :D

[sushi]

Originally posted by AhmedFaisal
Completely dumb. The G5 Xserve Upgrade is only 2 months away (max) and price wise the Xserve Cluster node is less expensive than a dual CPU client.

Ahmed,

How do you know the G5 Xserve upgrade is only 2 months away?

My guess is that you are speculating/wishing that this will happen. We could see another PowerBook situation...and wait and wait...

Anyhow, there might be another reason that they went with the G5 and that it had to do with funding and fiscal year expenditures. October 1st is fastly approaching.

Sushi

[sushi]

Originally posted by AidenShaw
Your wish for 128-bit won't be needed while any of us are alive - but I'm sure that the starships' Enterprise computers are more than 64-bit. It's silly.

Yeah, and 640K is enough for anyone! LOL!

No one knows the future direction of computers.

Today I have more RAM in my desktop computer than I had HD space ten years ago!

There are so many things that are changing the computer landscape and pushing the need for increased computing power. Will we reach a point where this stops? I doubt it.

Who knows, maybe we will see 128 bit computing sooner than anyone expects it. :D

Sushi

[whooleytoo]

Originally posted by AidenShaw
That's not a 64-bit application. For proof - AltiVec has been manipulating 128-bit chunks of data, but nobody calls the G4 a 128-bit system. G3/G4 and Pentium chips have 64-bit floating point, but nobody calls them 64-bit computers.

It's the width of the address pointer that matters, not the width of the largest datum.

The Integer registers are almost always used for address manipulation, therefore the width of the integer registers is always a limiting factor in the amount of addressable memory.

So when people say a 64bit chip = a chip with 64bit integers, they're pretty much saying it's a chip with 64 bit addressing.

Mike.

[AidenShaw]

Originally posted by whooleytoo
So when people say a 64bit chip = a chip with 64bit integers, they're pretty much saying it's a chip with 64 bit addressing.

True that (almost all) chips with 64-bit addressing also have 64-bit integers, converse is not true.

And, when Apple ships an operating system that uses 32-bits of those registers for addressing, it's shipping a 32-bit system (running on a 64-bit CPU).

To be precise - a programmer using a 64-bit system should be able see the value "8" when he runs a native program that does:

printf ("%d\n",sizeof(void *));

If he sees "4", then it's a 32-bit system - regardless of the addressing capability of the actual chips.

[AidenShaw]

Originally posted by sushi
Yeah, and 640K is enough for anyone! LOL!

No one knows the future direction of computers.

Fair enough.... Also, nothing requires that the next jump has to be another doubling of the address width - 80-bit or 96-bit addressing might be the way to go.



Originally posted by sushi
Today I have more RAM in my desktop computer than I had HD space ten years ago!

:) ... and the VRAM in my graphics card is more than the HD space in my first 486 - and I opted for the big 110MB disk!

[esheep2001]

Originally posted by AidenShaw
:) ... and the VRAM in my graphics card is more than the HD space in my first 486 - and I opted for the big 110MB disk!

Heh, heh! :-)

The L2 cache in my new G5 (512k) is 128 times the amount of RAM I had in my first computer!

e.

[whooleytoo]

Originally posted by AidenShaw
True that (almost all) chips with 64-bit addressing also have 64-bit integers, converse is not true.


Out of (genuine) curiosity, do you know a machine with 64 bit ints that doesn't support 64bit addressing?

(Talking about the hardware limitations only - since it's perfectly possible to build a 64bit machine and put an OS on it that only supports 16 or even 8 bit addressing!)

Mike.

[alandail]

Originally posted by AhmedFaisal
Regarding 1.:

The will be out within a reasonable timeframe.

Regarding 2.:

Get real. Before a university/government institution is allowed to sell any kind of hardware to anyone some years go by or the things die so badly its not worth repairing the thing. The newest Macs my university for example is allowed to sell at the moment are 7xxx series.

As for use anywhere else. You can bet that these machines will be stripped off anything not useful in a cluster node (unless they wen't totally bonkers when they placed the order). So they will have either a Rage 128 Graphics Card or none, same with optical drive, none or CD-ROM, I would even bet none for both and they do network installs and an absolute minimal HD (since a cluster node doesn't need much HD space). So if they want to ever use them outside the cluster they would have to upgrade them with new Graphics Card, Optical Drive and HD.
Cheers,

Ahmed

1 - clearly not soon enough to make the top 5 list this year, which was a criteria - not to make Apple look good, but to make Virginia Tech look good.

2 - suppose they already plan to replace the cluster with dual 3 GHz machines next fall so they will still be top 5. They very likely are getting machines with a minimum of a 160 gig hard drive, a CD-RW drive, and a Radeon 9600 Pro. Apple doesn't sell G5's without these 3. If the intent is to upgrade the cluster each year it makes a lot more sense to order desktops than XServes because of the resale to student value.

[AidenShaw]

Originally posted by whooleytoo
Out of (genuine) curiosity, do you know a machine with 64 bit ints that doesn't support 64bit addressing?

http://www.tommesani.com/images/P4Logo.jpg

The SSE2 SIMD unit supports 64-bit integers - even the Microsoft Visual C++ compiler will generate SSE2 64-bit integer instructions if told to. (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vccore/html/vclrfArchMinimumCPUArchitecture.asp)


There are enough oddball chips and features that one can often be nailed with a counter-example if one says something is always, or never or similar absolute terms.

If you fuzz it with an "almost" or a similar "wiggle-word", you can describe the common, commercially successful architectures without getting in trouble.

But, I agree with you that most microprocessors use the same ALU and registers for addressing and integer arithmetic, and therefore max integer size and virtual address are the same.

[Rocketman]

One of the posts with first or second hand information said VT went to Dell and HP with a deadline and they could not meet it. Apple could. This seems like a rational explanation to me, as school does start on a date specific and whatever taks the farm is being built for, is obviously for students to work with to some degree.

I initially assumed the farm would be disparate desktops linked by a network. I was wrong about that. They are putting these CPU's in a rack or series of racks as it turns out, possibly to reduce latency issues.

I suppose students could log onto particular CPU's remotely :)

What is NOT confusing to me aout this is:

1. G5 2x2 desktop CPU's are here today, not later.

2. Even if an x-serve G5 were to be released in say only 3 months, the school year would be 1/3 over by then and the release would likely have some other cripple due to temperature or density or some such and for a cluster, not offer a whole lot more functionality. I think we can assume an Apple sales rep actually gave them a realistic alternative timeframe if not specific details of the coming x-serve before this purchase decsion was made.

3. There are reasons to believe an Apple purchase was potentially a better short term choice for purely performance reasons. The speed specs are better than most current wintel alternatives, and at less electrical and temperature concerns, which were clearly large budget items for this cluster farm.

4. Once the software is written and compiled for G5 and to some extent for altivec, Apple and IBM (IBM adds tremendous credibility and lowers purchase risk) have clearly shown a roadmap to significantly faster and more capable broadband CPU's in the IMMEDIATE future, something no wintel processor can even begin to do.

5. Thus the processor vendor has legs, is more immediately available, uses less support resources, is likely as fast or faster, and offers lower purchase risk. Am I actually talking about Apple Computer, Inc here?

What's wrong with this picture? Apple stock up again today.

Rocketman

[BigJayhawk]

The info from the Virginia Tech posts states that there was a VERY SHORT timeline to get this going.

If Apple told VT to wait for the Xserve G5 to come out doesn't anyone else see that HP or Dell (or SOMEONE) could have filled the order before the Xserve G5's release?

(And before someone says "but the G5 is faster" -- that's irrelevant when you can just add more than 1100 to the cluster to attain the same speed. There are no Apple computer products on the top 500 yet. There ARE some Pentiums and Athlons, etc.)

[AidenShaw]

Originally posted by Rocketman
They are putting these CPU's in a rack or series of racks as it turns out, possibly to reduce latency issues.
...
The speed specs are better than most current wintel alternatives, and at less electrical and temperature concerns, which were clearly large budget items for this cluster farm.

The interconnect is InfiniBand, a high bandwidth (up to 10Gbps and more) low latency computer room solution.

As to power/performance - the Intel blade systems (such as IBM BladeCenter http://www-1.ibm.com/servers/eserver/bladecenter/ or HP ProLiant Blades http://h18004.www1.hp.com/products/servers/platforms/index-bl.html) have a very good power ratio - and support up to dual 3.06GHz Xeons in a 1/2 U effective space (14 dual CPU blades in a 7U chassis).

The PPC970 isn't too far from the Xeon for power consumption, close enough that the power used in the chassis is important.

If the VT application is AltiVec-friendly, then the G5s might be the better choice. Your other arguments, however, are not as strong.

And the Power Mac is a poor choice for space efficiency - you could get 84 to 168 3.06 GHz Xeons in a standard rack - compared to only 12 PPC970s.

Not only is space expensive, but high-speed low-latency interconnects often have rather limited lengths (InfiniBand over copper quotes 16 meters). "Server sprawl" can be a problem ;)

[AidenShaw]

Originally posted by BigJayhawk
There are no Apple computer products on the top 500 yet. There ARE some Pentiums and Athlons, etc.)

Actually, 122 of the top 500 are Intel-based - 100 Pentiums, and 22 Itanium and AMD systems.

#3 is a Xeon cluster, and there's another Xeon cluster and an Itanium cluster in the top 10.

[Rocketman]

Originally posted by AidenShaw
The interconnect is InfiniBand, a high bandwidth (up to 10Gbps and more) low latency computer room solution.

Your other arguments, however, are not as strong.

And the Power Mac is a poor choice for space efficiency - you could get 84 to 168 3.06 GHz Xeons in a standard rack - compared to only 12 PPC970s.

Not only is space expensive, but high-speed low-latency interconnects often have rather limited lengths (InfiniBand over copper quotes 16 meters). "Server sprawl" can be a problem ;)

Agreed which is why I initially incorrectly assumed they might use CPU's sitting in a student lab for normal daily use and just use spare processor cycles with either 1000BT ethernet or even IP over firewire 800 insterad of infiniband.

But it seems they are targeting some truly leading edge bandwidth applications. Not sure how high a duty cycle thay can maintain on this cluster farm, or if it is mainly for brag value.

Rocketman

Can we log in? :)

[AidenShaw]

Originally posted by Rocketman
it seems they are targeting some truly leading edge bandwidth applications. Not sure how high a duty cycle thay can maintain on this cluster farm, or if it is mainly for brag value.


I'm sure that Apple is looking forward to a place on the Top 500 !

The "duty cycle" shouldn't be a problem (except for building power failures and similar events).

It may be that there is never a time when all 1100 systems are working - these large clusters assume that some systems will be dead, and tolerate it. It's even possible that the design goal is 1000 systems, and the extra hundred are there to make sure that at least 1000 are always running.

Looking at it another way, if the MTBF (Mean Time Between Failure) of the Power Mac is 3 years - then statistically a system will fail every day.

Personally, I'd worry about memory errors. Compaq did a study that showed that one can expect a soft memory error at a rate of one error per gibibyte per year.

Since the VT cluster will have 4-8 GiB per system - that works out to one error every hour or two.

Would you want to have your computer giving you the wrong answer every hour?

Other manufacturers offer hot swap memory, RAID memory, "spare" memory and similar features. Apple doesn't even believe in ECC :mad:

[whooleytoo]

Originally posted by AidenShaw
http://www.tommesani.com/images/P4Logo.jpg

The SSE2 SIMD unit supports 64-bit integers - even the Microsoft Visual C++ compiler will generate SSE2 64-bit integer instructions if told to. (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vccore/html/vclrfArchMinimumCPUArchitecture.asp)


Well, when I said '64 bit ints' I meant 64 bit registers in the integer unit. (But kudos anyway for coming up with the counter example! ;-) In my experience, 64 bit integer units and 64 bit addressing (hardware) go hand in hand.

And I agree, the general rule about such generalisations is they're generally false!

Anway, apart from the memory addressing issue, I am genuinely how the G5 will shape up in 64bit integer math vs (say) the G4. How significant will be the speed increase, or will it be entirely offset by the need to move data of twice the width around... Has anyone seen any benchmarks of 64bit int code? (Mathematica perhaps??)

Mike.

[backspinner]

Originally posted by Rocketman
Apple stock up again today. :D stock markets are closed today (monday)

Originally posted by AidenShaw
gibibyte and how many is that?

[AidenShaw]

Originally posted by backspinner
gibibyte

and how many is that?

1,073,741,824 (or 2 to the 30th power)


See http://physics.nist.gov/cuu/Units/binary.html or http://members.optus.net/alexey/prefBin.html or http://www.theregister.co.uk/content/54/22734.html ...

[PHGN]

They very likely are getting machines with a minimum of a 160 gig hard drive, a CD-RW drive, and a Radeon 9600 Pro. Apple doesn't sell G5's without these 3.

If you order 1100 brand new machines in an attempt to create one of the top 5 clusters then Apple will do whatever you want.

Then they'll send some photographers round.


As to those who whine about having their machine delayed so VTcan get theirs:

A) 1100 machines is hardly going to cause much of a hold up compared to the total number ordered.

B) The VT machines are likely to be a custom spec and thus not come from the standard consumer supply.

C) The VT order was probably made before the G5 was announced -- they probably went round compaies to see who could meet the deadline. When they asked Apple (I'm guessing they expected X-Serve clusters) Apple said yes we have something special for you too. In other words their order was probably the very first so don't gripe about the fact that you won't get yours untill after them.

[eric67]

According to hardmac.com (english version of macbidouille.com), the G5 could be delivered first in Europe.... as Apple Assembly Line in Corck (Ireland) is designed for PowerMac.
let's wait and see

[shadowfax]

Originally posted by eric67
According to hardmac.com (english version of macbidouille.com), the G5 could be delivered first in Europe.... as Apple Assembly Line in Corck (Ireland) is designed for PowerMac.
let's wait and see apple manufactures its equipment in Europe? really?

[Mr. G4]

Or may be Steve wanted VT to set them up...films it in action and shows it off in Paris :)

Originally posted by ImAlwaysRight
So why in the world can't VT wait a month to receive their 1100 computers and instead the masses who ordered first receive their computers first? What is the rush? Like one month will make a difference to them. I'm sure it makes more of a difference to the 1100 or so individuals who are waiting on their PowerMacs because of this order. And you can't tell me VT placed their huge order the day these G5's were announced. It takes more than a few hours to receive approval for a purchase of that size. :mad:

ADDITION: What will win more people to Mac platform, actually SEEING friends, professionals, etc. using PowerMac G5's, or hearing news about 1100 G5's being used in a cluster? How many people are actually going to cluster 1100 computers? Probably none.

So to me it is better for Apple to get G5's in the hands of everyday people and make the cluster wait their turn. Also, Apple can still benefit from ANNOUNCING that VT has a huge order which would be fulfilled later in Sept. for a supercluster. So I don't understand why this is such a great move on Apple's part.

I would imagine this order was placed late July when Apple changed their shipping time from "August" to "7-10 weeks". As I said before, a purchase this size would not have been decided overnight.

[whooleytoo]

Originally posted by shadowfax
apple manufactures its equipment in Europe? really?

AFAIK, all desktops destined for the European market are assembled in Cork. I'd be surprised if the plant was supplying the US's needs as well - unless they needed extra help in keeping up with US demand?

Mike.

[iDONTsteal]

Originally posted by AidenShaw <<<AidenShaw is the bomb baby!>>>
1,073,741,824 (or 2 to the 30th power)


See http://physics.nist.gov/cuu/Units/binary.html or http://members.optus.net/alexey/prefBin.html or http://www.theregister.co.uk/content/54/22734.html ...


2 ^ 30 Gibi

vs.

10 ^ 9 Giga

I know... it was linked... but who wants to have to do that??!? <it's so exhausting!>

:p
:)

[alandail]

Originally posted by PHGN
[B]If you order 1100 brand new machines in an attempt to create one of the top 5 clusters then Apple will do whatever you want.



within reason - the G5 is the only serial ATA machine Apple sells - the 160 gig drive is the smallest serial ATA drive they offer. I highly doubt VT had Apple custom ordered 1100 smaller drives instead of going with the 160 gig drive Apple already uses in production. That'd just be more trouble than it's worth. maybe they cut out the optical drive and/or the video card to cut costs - it still doesn't make the machines hard to sell to students in a year.

I'm sure if VT goes to Apple and says we'd like to order 1100 of your latest machines to upgrade our cluster, Apple would also sell them superdrives and video cards for the machines they're replacing so VT can sell them to students.

If I were designing a cluster like this, I'd have to think strongly about designing it to be easily upgradable, and selling through the machines to students significantly cuts the effective upgrade costs.

I do agree with the other thing you said - these machines are going to have a negligible impact on the delivery of other machines.

[alandail]

VT couldn't wait another month. They clearly would have gone with a different system to meet their deadline had they been told they would have to wait for Apple. They can't change the start of the school year.

[Phil Of Mac]

Originally posted by alandail
2 - suppose they already plan to replace the cluster with dual 3 GHz machines next fall so they will still be top 5. They very likely are getting machines with a minimum of a 160 gig hard drive, a CD-RW drive, and a Radeon 9600 Pro. Apple doesn't sell G5's without these 3. If the intent is to upgrade the cluster each year it makes a lot more sense to order desktops than XServes because of the resale to student value.

This is a special order. Apple may very well sell G5's without these 3 if it's a special order.

That said, it would be expensive to build a brand new cluster every year. These things are supposed to last awhile :)

[XnavxeMiyyep]

If I were to get a G5, would I be able to cluster it with my G4?

[Phil Of Mac]

Originally posted by XnavxeMiyyep
If I were to get a G5, would I be able to cluster it with my G4?

I don't think there's any convenient "cluster your computers at home" system in any case.

[iDONTsteal]

I think to do so would be signifigantly more effort than easily justifiable.

However... people do silly things all the time for the sole purpose of seeing if they can. I'm sure there is some way.... but for Joe/Jill Average Person... unjustifiable effort

[legion]

Originally posted by Phil Of Mac
I don't think there's any convenient "cluster your computers at home" system in any case.

I've got Linux clustering at home (4 computers) Very Easy. (and free!!)

[AidenShaw]

What do you think a cluster is, and why would you think that it would be a useful thing for 2 computers at home?

[XnavxeMiyyep]

Originally posted by legion
I've got Linux clustering at home (4 computers) Very Easy. (and free!!) How'd you do it? Would it work for Macs?

Originally posted by AidenShaw
What do you think a cluster is, and why would you think that it would be a useful thing for 2 computers at home? A group of computers combined together to make them faster. And it would be useful as it gradually gets bigger when I buy more computers.

[AidenShaw]

Originally posted by XnavxeMiyyep
A group of computers combined together to make them faster. And it would be useful as it gradually gets bigger when I buy more computers.


But only software written explicitly for a cluster will see any improvement. The application has to break the workflow up into separate pieces, and send each piece to a separate node for processing, then put all the results together.

It won't make Photoshop or FCP or Quake or any of the programs that you likely are running now any faster.

[cocoa_nut]

Mac clustering is drop-dead easy:

1) Grab a demo of Pooch from:

http://daugerresearch.com/pooch/download.html

and install the demo on your systems.

2) Grab an app designed for parallel operation:

http://daugerresearch.com/fractaldemos/altivecfractalcarbon.html

and run it with Pooch.

3) There is no step three !!! ;)

cocoa_nut

[XnavxeMiyyep]

Originally posted by cocoa_nut
Mac clustering is drop-dead easy:

1) Grab a demo of Pooch from:

http://daugerresearch.com/pooch/download.html

and install the demo on your systems.

2) Grab an app designed for parallel operation:

http://daugerresearch.com/fractaldemos/altivecfractalcarbon.html

and run it with Pooch.

3) There is no step three !!! ;)

cocoa_nut But I should ethernet them together, right? And will it improve multitasking, too?

[cocoa_nut]

XnavxeMiyyep:

The machines do indeed need to talk to each other. The required speed of this communication channel is totally dependent on the nature of the parallel application you intend to run. Some programs can be "parallel-ized" such that very little data need be transmitted between the nodes performing the computations, while other codes are structured such that high-bandwidth interconnects are required to pass data back and forth. So, yes you will need to network the computers in some way to form the cluster. :)

cocoa_nut

[XnavxeMiyyep]

Originally posted by cocoa_nut
XnavxeMiyyep:

The machines do indeed need to talk to each other. The required speed of this communication channel is totally dependent on the nature of the parallel application you intend to run. Some programs can be "parallel-ized" such that very little data need be transmitted between the nodes performing the computations, while other codes are structured such that high-bandwidth interconnects are required to pass data back and forth. So, yes you will need to network the computers in some way to form the cluster. :)

cocoa_nut Ok. My Dad might give me his old G3 iMac, cuz i gave him a faster one. I'll test it with that.

[neilt]

Hey everyone....

Our cluster is no 65 on the list... and yes, the timing is very important. We only had it on for about a week when the deadline for the list came around. We didn't have anything optimized properly, and many of the nodes were mis-wired (ie, the management consoles are on 100Base-T, and the compute nodes are on GigE, but many of them were plugged into the 100 switches instead of the gig switches.
Anyway, we got on at 65, although i have been told that we are now benchmarking fast enough to be in the top 30...

Anyway, i mentioned this vt rumor to some of the guys at work and this is was their responses. Thought you might be interested:

---
This puzzles me - they will have approx 2200 x 2GHz CPUs and we
have 1024 x 2.? GHz CPUs so they will have approx twice the raw CPU
power we have._ Why then do they think they are going to get 5 times
the max throughput we get - 10 Tflops vs approx 2 TFlops?

Is someone on crack or is it all down to the interconnects
(Infiniband, whatever that is)?_ It seem incredible that all that
speedup would be simply due to their CPUs being 64bit while ours
are 32bit._ I'll be very interested to see how this project pans out
and whether their cluster ever gets even close to that 10 Tflop number
they have mentioned.

John (the pessimist)

---
and the response from our cluster admin:
---
The interconnects play a HUGE role in achieving those sorts of numbers. The latency with Gig E is much more than myrinet or any other highspeed interconnect._ I can achieve about 500 G/Flop with the 128 myrinet nodes. It takes over twice that number of Gig/E nodes to produce similar numbers. Also the G5 floating point performance is much higher than the Xeon processors. Also being able to address more memory via 64bit is adventageous as well._ The early Opteron benchmark numbers show this very well.

For your reference:
http://www.amd.com/us-en/Processors/ProductInformation/0,,30_118_8796_8800~70045,00.html
http://www.apple.com/powermac/

---
John's point is well taken but Virginia tech_will have much more options with a 64 bit system than people limited to 32 bit, re: memory access per_node._ However,_note that Cray is now using the AMD chip and a newly-developed multiCPU backplane to put Sandia Labs back into Top500_contention by creating a system_like TimeLogic and Paracel - 32 bit, standard cheap memory and 2G per CPU but very few interconnects, everything is on 16 enormous backplanes with full-speed access to the memory pipeline, and they have the potential to exceed 32K 32-bit_processors and 64 TB of RAM_per_system (Paracel was at 18K CPU's at their last iteration) making Cray the lowest cost per clock cycle at roughly $16M (3x our price) per system and potentially exceeding 20 TF (10x our performance and 2x Virginia Tech but more scalable)._ Also, the Cray system could be an Unreal server without modifications.

[AidenShaw]

Originally posted by neilt
Also being able to address more memory via 64bit is adventageous as well

Did you point out that OS X and YDL are 32-bit, and therefore applications cannot address more memory than Xeon systems?

[Rocketman]

Originally posted by AidenShaw
Did you point out that OS X and YDL are 32-bit, and therefore cannot address more memory than Xeon systems?

nielt makes some interesting points that highlight some of the statements about VT.

1. Very low latency interconnects and proximate co-location.

2. Maxed out memory capacity and speed.

3. Dualies on every box thus effectively lowering the average latency issue.

4. YDL which frankly is supportive of going 64 bit on as many drivers as the "customer" is willing to assist with, and wana bet how much VT is willing to assist YD? Not to mention any open source aspects of OSX.

What is particularly interesting is the G5 has been on the street for a little over a week, the dualies have not arrived anywhere visible yet, but 64 bit versions of Panther have been seeded to "some folks". One wonders if places like VT will get 64 bit Panther in time for next year's list.

They certainly will get G5's, hyperinterconnect and maxed out internal resources in time for THIS YEAR.

10 Teraflops? Probably not. But well over 4 and approaching 6? Yep.

Rocketman

[AidenShaw]

Originally posted by Rocketman
....but 64 bit versions of Panther have been seeded to "some folks". One wonders if places like VT will get 64 bit Panther in time for next year's list.

I haven't seen any news of "64-bit versions of 10.3" - please add a link to that info...

As far as VT and 64-bit, unless their application needs more than 4GiB per task, there's no benefit to going to 64-bit for them. Sticking with 32-bit could be the right thing for their apps.

[AidenShaw]

Originally posted by XnavxeMiyyep
But I should ethernet them together, right? And will it improve multitasking, too?

It won't help multi-tasking, and it won't help any of your existing apps - they haven't been written for clustering.

It will help the programs that you download from the pooch site, though. Lots of fun with fractal toys....

[alandail]

I wonder how much the new IBM compilers will help when they benchmark this cluster.

[cr2sh]

Originally posted by AidenShaw
unless their application needs more than 4GiB per task, there's no benefit to going to 64-bit for them.

Is this true? Many of my applications do not require 4Gib (a gross understatement), so I have no need for 64bit?

[MadMan]

FWIW - From Macintouch:

Following up on the note we published in late July, a MacInTouch reader writes:
Several weeks ago I sent word to you about Virginia Tech building a supercomputer with G5s; well, the news is out and watching the forums I would like to clear up some things that people are saying.
_ [...] The cluster will indeed run Mac OS X, not Linux as many people are saying. They have purchased 1,100 dual 2Gigs, with 4 gigs of RAM. They were purchased with no special discount from Apple. It is expected to be the 3rd or 4th fastest supercomputer in the world.

[ExtremeVT]

No, feel free to apply for the Spring semester.

[AidenShaw]

Originally posted by cr2sh
Many of my applications do not require 4Gib (a gross understatement), so I have no need for 64bit?

In almost all cases you won't benefit from 64-bit until you *need* more than 4 GiB of RAM, and you're running large applications that need the 4 GiB all to themselves.

If you have 512MiB or 1 GiB of RAM on your system - almost certainly 64-bit will be of no use.

If you run many small applications, 32-bit should be OK.

10.2.7 / 10.3 are 32-bit, yet they can run about sixteen 2 GiB applications at the same time on a system with 8 GiB or RAM - each app gets its own 2 GiB. You don't need a 64-bit operating system or a 64-bit chip for this - Pentiums and G4 chips support up to 64 GiB of RAM on a 32-bit chip.

You'll want the PPC970 because it's fast - but not because it is 64-bit. It shouldn't matter that OS X doesn't support 64-bit.

-as

ps: There's one minor exception to this - the 64-bit chip can do native 64-bit integer arithmetic even when it is running with 32-bit addresses. A very small number of programs spend a lot of time with large integers, and those programs could see an improvement on the PPC970. How much improvement would depend on the program, and on how much of its time is in 64-bit integer code.

[Smurfman]

Originally posted by DrGonzo
why would you buy 1100 first revision g5s? I understand they want the machines ASAP... but still to spend at least $2mil on a bunch of machines that are brand spanking new with no small scale usage and certainly no large scale, and also were, for the first time, mass produced... just doesn't seem like a great way to do things. Then again, maybe this cluster isn't that important.

One of the top 5 fastest supercomputers in the world and you're saying it might not be that important?!?

Va Tech knows what they're doing in this respect. As other's have stated, the machines can one day be replaced with faster models and then passed down to another department for individual workstation use. They are saving a boatload of $$ on this deal compared to other offerings. I'm sure the total cost of ownership for these machines knocks-the-socks-off anything else.

BTW... the money for this project did come from a grant.

[Ambrose Chapel]

September 2 - 17:21 EDT__ Virginia Tech confirmed Tuesday that it will use 1,100 Power Mac G5s as part of a supercomputer cluster now under construction. "The new cluster is designed to make its way into the rankings of the world's largest supercomputers, a list that currently has no Macs," reports CNET News.com. "Virginia Tech will use the cluster to perform research on nanoscale electronics, chemistry, aerodynamics, molecular statics, computational acoustics and molecular modeling, among other tasks." The university said it has been working with Apple for months to set up the cluster. Interestingly, the school said dual 2GHz machines started coming off the manufacturing lines last month. The news was first reported by Think Secret.
http://www.macminute.com/2003/09/02/vt

[pjhornak]

Originally posted by AidenShaw
In almost all cases you won't benefit from 64-bit until you *need* more than 4 GiB of RAM, and you're running large applications that need the 4 GiB all to themselves.

If you have 512MiB or 1 GiB of RAM on your system - almost certainly 64-bit will be of no use.

If you run many small applications, 32-bit should be OK.

10.2.7 / 10.3 are 32-bit, yet they can run about sixteen 2 GiB applications at the same time on a system with 8 GiB or RAM - each app gets its own 2 GiB. You don't need a 64-bit operating system or a 64-bit chip for this - Pentiums and G4 chips support up to 64 GiB of RAM on a 32-bit chip.

You'll want the PPC970 because it's fast - but not because it is 64-bit. It shouldn't matter that OS X doesn't support 64-bit.

-as

yah learn somethin' new everyday...

ps: There's one minor exception to this - the 64-bit chip can do native 64-bit integer arithmetic even when it is running with 32-bit addresses. A very small number of programs spend a lot of time with large integers, and those programs could see an improvement on the PPC970. How much improvement would depend on the program, and on how much of its time is in 64-bit integer code.