You mean Darwin? Even if some of the additional OS X specific system services are required, I'm sure they can work something out. No need to be so GUI heavy. On a related note, why would such a task require a graphics card? There's a lot of room for tweaking.
Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
More Virginia Tech Cluster Details
- Thread starter MacRumors
- Start date
- Sort by reaction score
You mean Darwin? Even if some of the additional OS X specific system services are required, I'm sure they can work something out. No need to be so GUI heavy. On a related note, why would such a task require a graphics card? There's a lot of room for tweaking.
Estimated Performance:
I just felt funny and thought I'd calculate the Rpeak-Value with the simple equation
FPops/Cycle * MHz * Number of CPUs
(try it, it works with all the Top500 Rpeak-Values! Xeon and Alpha have 2 FPops/cycle, Power3/4 and Itanium have 4!)
..we end up with 17.6 TFLOPS Rpeak (4 FPops/Cycle * 2 GHz * 1100 Macs * 2 CPUs/Mac), and that is JUST the FPUs without any Altivec!
yes, the G5 has, just like Power3/4 4 FPops/cycle, even though it only has 2 FPUs, because the PPCs combined Multiply-Add-Op accounts for 2 FPops! ;-)
This puts it pretty close to the Rpeak of the 8192 1.25GHz Alpha-Cluster on Place 2 (20.4 TFLOPS)! Now we just need to wait how much Rmax they get out of the Rpeak! ;-)
The whole thing becomes even more funny if you consider Altivec-Floatingpoint in the Equation! Even though it can handle 4 values in one cycle Altivec unfortunately can only do 32bit Floats (single precision), but let's just assume for fun we have/need 32bit Floats only: (4 FPops/Cycle + 8 FPops/Cycle (Altivec-FPMul-Add!)) * 2 GHz * 1100 Macs * 2 CPUs/Mac = 52.8 TFLOPS!!!!
Just for comparison: The so far unchallenged Earth Simulator at #1 "only" has 40.9 TFLOPS Rpeak!!
I'm not f*cking kidding! ;-) Ofcouse it's only a theoretical value as you just can't feed 2 FPU-MulAdds and 1 Altivec-MulAdd (plus Data!) constantly every single cycle, but remember: All the Rpeak-Values are theoretical maxima! ;-) It's just the question how close you can get with the actual Rmax-Linpack-Resuit! But if you look at the Rpeak there's certainly alot of potential in this Cluster!
Now just let someonce like the NASAs Craig Hunter and his Jet3D or anyone able to write good vectorcode onto that thing and you can watch it SERIOUSLY fly! ;-)
Now, does anyone know how much of linpack needs 64bit? ;-) And one question remains: Does this thing belong to the Microprocessor-Class or the Vectorprocessor-Class (like the japanese ones or Crays!)?
P.S.: Just for information: A single G5 2GHz has 24 GFLOPS theoretical Peak! 8 GFLOPS from the FPUs and 16 GFLOPS from Altivec-FP! If you don't believe me, read here, Page 4!
I just felt funny and thought I'd calculate the Rpeak-Value with the simple equation
FPops/Cycle * MHz * Number of CPUs
(try it, it works with all the Top500 Rpeak-Values! Xeon and Alpha have 2 FPops/cycle, Power3/4 and Itanium have 4!)
..we end up with 17.6 TFLOPS Rpeak (4 FPops/Cycle * 2 GHz * 1100 Macs * 2 CPUs/Mac), and that is JUST the FPUs without any Altivec!
yes, the G5 has, just like Power3/4 4 FPops/cycle, even though it only has 2 FPUs, because the PPCs combined Multiply-Add-Op accounts for 2 FPops! ;-)
This puts it pretty close to the Rpeak of the 8192 1.25GHz Alpha-Cluster on Place 2 (20.4 TFLOPS)! Now we just need to wait how much Rmax they get out of the Rpeak! ;-)
The whole thing becomes even more funny if you consider Altivec-Floatingpoint in the Equation! Even though it can handle 4 values in one cycle Altivec unfortunately can only do 32bit Floats (single precision), but let's just assume for fun we have/need 32bit Floats only: (4 FPops/Cycle + 8 FPops/Cycle (Altivec-FPMul-Add!)) * 2 GHz * 1100 Macs * 2 CPUs/Mac = 52.8 TFLOPS!!!!
Just for comparison: The so far unchallenged Earth Simulator at #1 "only" has 40.9 TFLOPS Rpeak!!
I'm not f*cking kidding! ;-) Ofcouse it's only a theoretical value as you just can't feed 2 FPU-MulAdds and 1 Altivec-MulAdd (plus Data!) constantly every single cycle, but remember: All the Rpeak-Values are theoretical maxima! ;-) It's just the question how close you can get with the actual Rmax-Linpack-Resuit! But if you look at the Rpeak there's certainly alot of potential in this Cluster!
Now just let someonce like the NASAs Craig Hunter and his Jet3D or anyone able to write good vectorcode onto that thing and you can watch it SERIOUSLY fly! ;-)
Now, does anyone know how much of linpack needs 64bit? ;-) And one question remains: Does this thing belong to the Microprocessor-Class or the Vectorprocessor-Class (like the japanese ones or Crays!)?
P.S.: Just for information: A single G5 2GHz has 24 GFLOPS theoretical Peak! 8 GFLOPS from the FPUs and 16 GFLOPS from Altivec-FP! If you don't believe me, read here, Page 4!
Buy low or wait! (and this is GOOD news)
If this is true I blame Moto, not Apple--but I sure wouldn't give my money to Apple for a G4 in October. That's too close to when I'd expect a G5!
But if you NEED a laptop before then, the current PowerBooks are still good... get the best deal you can and buy at the bottom-end: the 12" or an iBook. You'll have a nice extra-portable companion for your larger-size PB G5 later on.
A shame, but it sounds like there's nothing we or Apple can do... until the PBG5 is reality. Meanwhile, I'd rather own a 1 GHz PowerBook than any speed of Windows laptop!
In any case, I was always planning to wait for a G5, so this is GOOD news, if it makes Apple rush to make that happen sooner!
Apple will lose a LOT of laptop revenue from this, and some short-term market share too. But Apple can easily survive that.
If this is true I blame Moto, not Apple--but I sure wouldn't give my money to Apple for a G4 in October. That's too close to when I'd expect a G5!
But if you NEED a laptop before then, the current PowerBooks are still good... get the best deal you can and buy at the bottom-end: the 12" or an iBook. You'll have a nice extra-portable companion for your larger-size PB G5 later on.
A shame, but it sounds like there's nothing we or Apple can do... until the PBG5 is reality. Meanwhile, I'd rather own a 1 GHz PowerBook than any speed of Windows laptop!
In any case, I was always planning to wait for a G5, so this is GOOD news, if it makes Apple rush to make that happen sooner!
Apple will lose a LOT of laptop revenue from this, and some short-term market share too. But Apple can easily survive that.
Re: Re: More Virginia Tech Cluster Details
Probably a large portion of the hardware was sold to them at, or slightly above cost. And you're forgetting that they have an abundance of cheap/free labor, students.
This may not necessarily be true. When it comes to clusters in a university setting, it has been reported that MS has provided the school with free licenses. I'm not defending MS but the amount that companies/universities pay for licensing in certain situations can be trivial to non-existent.
Re: Re: Re: More Virginia Tech Cluster Details
I've been in this situation before. The hardware is rarely done at cost. But even assuming they got the Macs at $2,500 each this is 2.75 million just for the Macs. (This is $2,314 off [almost half off] of the standard educational pricing for a 4 GB 2.0 GHz dual G5, and this is $2,849 off the list price [much more than half off].) Assuming similar deals on everything else (at least half off retail) will still bring the initial cost of an "all in" system (computers, networking cards, routers, cabling, racks, cooling system, high end UPS [you don't run these types of machines right off the public utilities], system administrative setup, etc., etc.) will certainly total over $4 million.
I was a grad student a long, long time ago ($9600 a year for 30+ [often 50+] hours a week as a research fellow; research assistants made even less back then). I don't know what grad students make these days, but I doubt it is much less than $18,000 a year. (You can go flip burgers for not much less than that.) If we assume an average of $18,000 a year over all the staff (under grads, grad students, professional staff [of which there must be 1 or 2 at least], administrative staff, etc. -- clearly some will be paid much more than $18k a year, e.g. the professional staff, some will be paid less) then the minimum body count I mentioned above for 24/7/365 support (15 bodies) will cost over $270,000 a year just for personnel. This does not take into account the cost of maintenance, electricity to run the systems and the cooling systems, etc. Even if I assume free hardware replacement parts (some of these 1,100 Macs WILL fail within 5 years) and free software and software upgrades for 5 years and I assume absolutely no cost indexing for inflation over the next 5 years, this is still over the $5.2 million budget.
Thus I still cannot believe a $5.2 million budget to include the initial system and operations for 5 years. The numbers just don't add up.
I've been in this situation before. The hardware is rarely done at cost. But even assuming they got the Macs at $2,500 each this is 2.75 million just for the Macs. (This is $2,314 off [almost half off] of the standard educational pricing for a 4 GB 2.0 GHz dual G5, and this is $2,849 off the list price [much more than half off].) Assuming similar deals on everything else (at least half off retail) will still bring the initial cost of an "all in" system (computers, networking cards, routers, cabling, racks, cooling system, high end UPS [you don't run these types of machines right off the public utilities], system administrative setup, etc., etc.) will certainly total over $4 million.
I was a grad student a long, long time ago ($9600 a year for 30+ [often 50+] hours a week as a research fellow; research assistants made even less back then). I don't know what grad students make these days, but I doubt it is much less than $18,000 a year. (You can go flip burgers for not much less than that.) If we assume an average of $18,000 a year over all the staff (under grads, grad students, professional staff [of which there must be 1 or 2 at least], administrative staff, etc. -- clearly some will be paid much more than $18k a year, e.g. the professional staff, some will be paid less) then the minimum body count I mentioned above for 24/7/365 support (15 bodies) will cost over $270,000 a year just for personnel. This does not take into account the cost of maintenance, electricity to run the systems and the cooling systems, etc. Even if I assume free hardware replacement parts (some of these 1,100 Macs WILL fail within 5 years) and free software and software upgrades for 5 years and I assume absolutely no cost indexing for inflation over the next 5 years, this is still over the $5.2 million budget.
Thus I still cannot believe a $5.2 million budget to include the initial system and operations for 5 years. The numbers just don't add up.
what i thought was interesting was 'beta' version of x...
which to me answers all of the questions people have about x and clustering and cutting out the gui... i can't help but imagine this as a supercharged version of x that is specifically designed to work in a cluster environment...
matt
<edit>
well re-reading seems to make it sound like just the beta of panther, but i still can imagine a homegrown brew of x for clustering...
of course, that would be os x server.
hmm.
i'm gonna stop talking.
now.
which to me answers all of the questions people have about x and clustering and cutting out the gui... i can't help but imagine this as a supercharged version of x that is specifically designed to work in a cluster environment...
matt
<edit>
well re-reading seems to make it sound like just the beta of panther, but i still can imagine a homegrown brew of x for clustering...
of course, that would be os x server.
hmm.
i'm gonna stop talking.
now.
GUI-less OS X
i know that you can run os x in command-line mode
just change user settings to require the user name and password ( i can't remember the exact wording or which pref pane it's in: i'm not at my mac right now).
then log out (apple-shift-q)
at the login screen type >console in the user box and hit return
you will be taken to a black screen (what looks like Teminal maximized to your whole screen) and asked for a user name and password (you can't backspace if you mess up on the password, FYI).
to return to the login screen, type exit and wait a second
i do this all the time just for the fun of it ("woah! your mac looks like it's running DOS!").
i know that you can run os x in command-line mode
just change user settings to require the user name and password ( i can't remember the exact wording or which pref pane it's in: i'm not at my mac right now).
then log out (apple-shift-q)
at the login screen type >console in the user box and hit return
you will be taken to a black screen (what looks like Teminal maximized to your whole screen) and asked for a user name and password (you can't backspace if you mess up on the password, FYI).
to return to the login screen, type exit and wait a second
i do this all the time just for the fun of it ("woah! your mac looks like it's running DOS!").
Re: Re: Re: Re: More Virginia Tech Cluster Details
You seem to be making a wrong assumption here.
The $5.2 million does not include the university's pre-existing operating expenses. The $5.2 million is how much EXTRA that they will have to pay for this new system; the $5.2 million is the amount of additional expenses above what they would be paying if they did not have the new G5 cluster.
The university may have to hire additional employees for the new system, but nowhere near the numbers that you were suggesting. The univeristy already has an extensive staff of employees maintaining other computer systems 24/365 (btw, why do you include a 7 ? ) who can also help monitor and maintain the new cluster without any extra expense to the university. Existing university professors who will be using this cluster will also be providing some of the support at no extra cost to the university. Thus the only payroll expenses in the $5.2 million figure are related to additional new hire positions.
You seem to be making a wrong assumption here.
The $5.2 million does not include the university's pre-existing operating expenses. The $5.2 million is how much EXTRA that they will have to pay for this new system; the $5.2 million is the amount of additional expenses above what they would be paying if they did not have the new G5 cluster.
The university may have to hire additional employees for the new system, but nowhere near the numbers that you were suggesting. The univeristy already has an extensive staff of employees maintaining other computer systems 24/365 (btw, why do you include a 7 ? ) who can also help monitor and maintain the new cluster without any extra expense to the university. Existing university professors who will be using this cluster will also be providing some of the support at no extra cost to the university. Thus the only payroll expenses in the $5.2 million figure are related to additional new hire positions.
Re: Re: Re: Re: Re: More Virginia Tech Cluster Details
This may be true, but I think that it is far more likely that there was an error in the reporting. It seems to me that even accounting for any discounts on the computers and hardware, the cluster itself would cost around $5 million minus labor. If we assume labor and maintenance is free (which it won't be, most clusters have a 24/7 devoted just to replacing downed nodes), we are still left with the additional operating expenses, power and cooling, which can get quite sizable (and that's ignoring the building space). Remember, Infiband cards and switches, though cheaper than other solutions, still will still easily run over $1 million to cluster. I don't expect Apple to be selling "at cost"--I've never heard of a single cluster being built that way.
My guess is that the $5.2 million is the cost of the hardware only or an expected "net" five-year cost. The actual five-year "all-in" cost would be higher, but that doesn't mean the university has to absorb all that. Heck, with the rates that NEC EarthSimulator charges, they could maybe even make a profit off of licensing time on the cluster.
This may be true, but I think that it is far more likely that there was an error in the reporting. It seems to me that even accounting for any discounts on the computers and hardware, the cluster itself would cost around $5 million minus labor. If we assume labor and maintenance is free (which it won't be, most clusters have a 24/7 devoted just to replacing downed nodes), we are still left with the additional operating expenses, power and cooling, which can get quite sizable (and that's ignoring the building space). Remember, Infiband cards and switches, though cheaper than other solutions, still will still easily run over $1 million to cluster. I don't expect Apple to be selling "at cost"--I've never heard of a single cluster being built that way.
My guess is that the $5.2 million is the cost of the hardware only or an expected "net" five-year cost. The actual five-year "all-in" cost would be higher, but that doesn't mean the university has to absorb all that. Heck, with the rates that NEC EarthSimulator charges, they could maybe even make a profit off of licensing time on the cluster.
No xserves in a while?
This must mean that Apple is not planning to release any g5 Xserves in a while.
Surely the cluster would have been better with xserves than with PowerMacs. It would be smaller, easier to manage, and not contain unnecessary graphics cards.
Surely, Apple would also have prefered them to use Xserves and wouold probably have told Va Tech if they had them coming soon.
No Xserves in a good while?!
This must mean that Apple is not planning to release any g5 Xserves in a while.
Surely the cluster would have been better with xserves than with PowerMacs. It would be smaller, easier to manage, and not contain unnecessary graphics cards.
Surely, Apple would also have prefered them to use Xserves and wouold probably have told Va Tech if they had them coming soon.
No Xserves in a good while?!
Re: Re: Re: Re: Re: More Virginia Tech Cluster Details
I was only talking about EXTRA expenses -- not pre-existing operating expenses. I assumed they already had a building with the capability to install a 1,100 node cluster. I assumed they are not adding things like additional connectivity to the Internet or Internet2. I assumed they were not adding incidental things like additional security systems/security personnel. I assumed they are not adding things like administrative/accounting overhead to track and bill users for their usage of time on the cluster. If you get into the nitty-gritty details of a major installation like this there are literally thousands of little things that individually cost, on average, only a few hundred dollars a year but add up to several tens or hundreds of thousands of dollars in total. I have purposely not included any of this stuff in order to come up with what I consider the absolute lowest theoretical cost over five years. This cost is still well above $5.2 million.
Hiring 15 people is not a large number. In reality when accounting for undergrads and grad students the total number of paid people working on/supporting the system over the course of any given year will almost definitely exceed 15.
Maintaining other compute systems? You mean they already have tasks to do? Then how are they going to maintain/support a huge 1,000-1,100 node cluster? For something this size there are always (and I do mean always) dedicated support personnel. The university will have to hire people (whether they are grad students or computer proffessionals does not matter in this discussion -- whether they already work for the university or are brought in from Los Alamos does not matter for this discussion -- the fact is that the university will have to create new, dedicated labor slots for this machine).
Bye-the-bye, I include the "7" out of habit from the old days. That is the way I learned it way back when (24 hours a day, seven days a week, 365 days a year). These are the fixed numbers. Since the monthly number changes month to month it is not included.
Users don't typically maintain on these large systems. The university professors will want to spend their time relative to the machine designing problems the computer can solve (developing theories, writing the algorithms, writing and debugging the software, etc.). Certainly a small subset of these professors will have some type of administrative responsibility over this system, but it will be rare to find the professors in there troubleshooting and repairing a failed node or interconnect.
That's all I was referring to. It does not matter if the new hire is a grad student or a secretary. The slot for this system is new to the university thus it is additional dollars. (Additionally, it does not matter if the secretary comes over from another department within the university. It is still a new slot for this system and most likely the university will have to replace him/her in the slot s/he moved from.)
My best guess is that there won't be any G5 XServes until January at the earliest. The university using stock PowerMac G5s supports this guess.
The university wanted to be listed in the next Top 500 listing. This means they have to have the thing up and running in October of this year. If no XServes until 2004 then they had to use stock PowerMacs. Apple is not going to push/hasten development of a new XServe for a single 1,100 order. It is just not cost effective, and hastening development of a new product often comes back and bites you in the form of unanticipated bugs.
I was only talking about EXTRA expenses -- not pre-existing operating expenses. I assumed they already had a building with the capability to install a 1,100 node cluster. I assumed they are not adding things like additional connectivity to the Internet or Internet2. I assumed they were not adding incidental things like additional security systems/security personnel. I assumed they are not adding things like administrative/accounting overhead to track and bill users for their usage of time on the cluster. If you get into the nitty-gritty details of a major installation like this there are literally thousands of little things that individually cost, on average, only a few hundred dollars a year but add up to several tens or hundreds of thousands of dollars in total. I have purposely not included any of this stuff in order to come up with what I consider the absolute lowest theoretical cost over five years. This cost is still well above $5.2 million.
Hiring 15 people is not a large number. In reality when accounting for undergrads and grad students the total number of paid people working on/supporting the system over the course of any given year will almost definitely exceed 15.
Maintaining other compute systems? You mean they already have tasks to do? Then how are they going to maintain/support a huge 1,000-1,100 node cluster? For something this size there are always (and I do mean always) dedicated support personnel. The university will have to hire people (whether they are grad students or computer proffessionals does not matter in this discussion -- whether they already work for the university or are brought in from Los Alamos does not matter for this discussion -- the fact is that the university will have to create new, dedicated labor slots for this machine).
Bye-the-bye, I include the "7" out of habit from the old days. That is the way I learned it way back when (24 hours a day, seven days a week, 365 days a year). These are the fixed numbers. Since the monthly number changes month to month it is not included.
Users don't typically maintain on these large systems. The university professors will want to spend their time relative to the machine designing problems the computer can solve (developing theories, writing the algorithms, writing and debugging the software, etc.). Certainly a small subset of these professors will have some type of administrative responsibility over this system, but it will be rare to find the professors in there troubleshooting and repairing a failed node or interconnect.
That's all I was referring to. It does not matter if the new hire is a grad student or a secretary. The slot for this system is new to the university thus it is additional dollars. (Additionally, it does not matter if the secretary comes over from another department within the university. It is still a new slot for this system and most likely the university will have to replace him/her in the slot s/he moved from.)
My best guess is that there won't be any G5 XServes until January at the earliest. The university using stock PowerMac G5s supports this guess.
The university wanted to be listed in the next Top 500 listing. This means they have to have the thing up and running in October of this year. If no XServes until 2004 then they had to use stock PowerMacs. Apple is not going to push/hasten development of a new XServe for a single 1,100 order. It is just not cost effective, and hastening development of a new product often comes back and bites you in the form of unanticipated bugs.
lots of new supercomputers
http://news.com.com/2100-1010_3-5071318.html
...The University of Utah will set up a supercomputer that's based on the Opteron processor from Advanced Micro Devices
...Last month, Los Alamos National Laboratory chose Linux Networx to build two large computing clusters that are based on Opteron. "Lightning," the larger of the two clusters, will contain 2,816 Opteron processors
http://news.com.com/2100-1010_3-5071318.html
...The University of Utah will set up a supercomputer that's based on the Opteron processor from Advanced Micro Devices
...Last month, Los Alamos National Laboratory chose Linux Networx to build two large computing clusters that are based on Opteron. "Lightning," the larger of the two clusters, will contain 2,816 Opteron processors
Re: lots of new supercomputers
Yes, and they are claiming the initial cost for 1,000 nodes is $2 million.
I was at the UofU back when they got a "super" a full blown IBM 3090-600VF -- and very loosely associated with the machine. It was a $22 million dollar set up ["list price" on everything] that they did a "special deal" with IBM for and got it for only $2 million -- or so they said. In reality, once all the costs were added up, the cost was almost 5 times that. Still a good deal, but no where near the cost quoted widely to the media at the time. [Additionally, it was set up in the business department -- not the computer science department or any other department of the college of engineering, nor was it set up on any department of the college of science. Go figure.]
Having that experience with the UofU many years ago is one of the many reasons why I don't believe the $5.2 million Virginian Tech five year number. Nor do I believe the UofU $2 million number.
There are always hidden costs they don't talk about that sometimes are as much as 4 to 5 times the quoted amount.
Yes, and they are claiming the initial cost for 1,000 nodes is $2 million.
I was at the UofU back when they got a "super" a full blown IBM 3090-600VF -- and very loosely associated with the machine. It was a $22 million dollar set up ["list price" on everything] that they did a "special deal" with IBM for and got it for only $2 million -- or so they said. In reality, once all the costs were added up, the cost was almost 5 times that. Still a good deal, but no where near the cost quoted widely to the media at the time. [Additionally, it was set up in the business department -- not the computer science department or any other department of the college of engineering, nor was it set up on any department of the college of science. Go figure.]
Having that experience with the UofU many years ago is one of the many reasons why I don't believe the $5.2 million Virginian Tech five year number. Nor do I believe the UofU $2 million number.
There are always hidden costs they don't talk about that sometimes are as much as 4 to 5 times the quoted amount.
Re: lots of new supercomputers
I like the idea that the supercomputer market has the competition necessary to keep the vendors on their toes and the prices "reasonable". The next time a university is ready to set up a supercomputer facility, they should be evaluating just how these facilities have fared.
Cluster Details
Tonight was the second night of orientation sessions for volunteers helping to set up the cluster. VT is actually recieving 1105 computers, 5 of them being single processor machines for administration purposes I would assume.
For those of you wondering why it is going to take so long to set up the cluster, here is the answer: First off, they are only unpacking 15 computers at a time, checking for DOAs (hopefully there will be none) installing the InfiniBand cards, and then moving them to the rack room to be placed in the rack. Secondly, the only cables run so far are the power cords. That leaves two InfiniBand Fibers and Gigabit Ethernet still to be put in place. AND the reall kicker is that the computers aren't coming in all at once, instead they will arrive on three different days.
Tonight was the second night of orientation sessions for volunteers helping to set up the cluster. VT is actually recieving 1105 computers, 5 of them being single processor machines for administration purposes I would assume.
For those of you wondering why it is going to take so long to set up the cluster, here is the answer: First off, they are only unpacking 15 computers at a time, checking for DOAs (hopefully there will be none) installing the InfiniBand cards, and then moving them to the rack room to be placed in the rack. Secondly, the only cables run so far are the power cords. That leaves two InfiniBand Fibers and Gigabit Ethernet still to be put in place. AND the reall kicker is that the computers aren't coming in all at once, instead they will arrive on three different days.
One more thing
Predictions on resales to students seem to be correct. All boxes, keyboards, and mice are being stored for the update to "a new form factor." I would say that is pretty good hint towards an upgrade path to include Xserve G5s or whatever may come after that.
The building these things are going in already has security for it, so no exense there. Plus the people in charge of running these things are current employees of VT. At most they may hire one or two more people to assist, but i see this thing being staffed mostly by grad students.
And a word on what they will be running on these machines. It will indeed be OS X with custom Fortran and C programs in addition to a couple commercial packages.
Predictions on resales to students seem to be correct. All boxes, keyboards, and mice are being stored for the update to "a new form factor." I would say that is pretty good hint towards an upgrade path to include Xserve G5s or whatever may come after that.
The building these things are going in already has security for it, so no exense there. Plus the people in charge of running these things are current employees of VT. At most they may hire one or two more people to assist, but i see this thing being staffed mostly by grad students.
And a word on what they will be running on these machines. It will indeed be OS X with custom Fortran and C programs in addition to a couple commercial packages.
Re: One more thing
Note to self: Cancel plan to wander in and walk off with a couple of Power Macs under each arm.
Have the specific projects been announced, i.e., the purposes of these programs?
Re: Re: lots of new supercomputers
My guess is that is the node cost and doesn't cover any of the switching hardware, let alone the labor, etc. I found the article had an interesting reference to the Angstrom superblades which I never heard of before.
Their website is pretty scarce on information, but the density is achieved by stacking the blade centers back to back (apparently the cooling is bidirectional). That's because the rack density is not very good (9U unit holds 13 blades). The blades can carry 2 Opterons with 4 DIMM bays (there are no specs on the actual blades, no specs on the HD or integrated video, etc).
I was curious if anybody knows anything about these things since their website is highly uninformative (even the prices are missing). For instance, are the power supplies hotswappable/redundant? Are the fans hotswappable/redundant? How can you hotswap when the units are back to back? Do they have a fiberchannel interface? Can you do anything other networking than serial/Gig-E? etc. It seems like not because they cut the length of the rack in half.
All in all, Utah is basically ordering a 1000 node BladeCenter from a company with no previous blade experience (they're ultra dense systems were simply two dual Athlon mobos crammed into a 1U case). I thought that nobody would take on a strategy as risky as VATech (building a cluster from an unreleased computer), but I was wrong.
If they were both finished today, I think the Utah system would have a higher Rpeak (I can't be sure since I'm too lazy to look up everything). Rmax would be lower due to bandwidth limitations of the blades. Even the former would be reversed if someone got a version of double precision Linpack (the test for the Top100) to work with Altivec optimizations (an 4x increase)--there is some notes about Altivec optimized single precision, but none about double. Is that a limitation of the G4 Altivec and does this limitation carry over to the G5 VMX? Anyone?
Take care,
terry
My guess is that is the node cost and doesn't cover any of the switching hardware, let alone the labor, etc. I found the article had an interesting reference to the Angstrom superblades which I never heard of before.
Their website is pretty scarce on information, but the density is achieved by stacking the blade centers back to back (apparently the cooling is bidirectional). That's because the rack density is not very good (9U unit holds 13 blades). The blades can carry 2 Opterons with 4 DIMM bays (there are no specs on the actual blades, no specs on the HD or integrated video, etc).
I was curious if anybody knows anything about these things since their website is highly uninformative (even the prices are missing). For instance, are the power supplies hotswappable/redundant? Are the fans hotswappable/redundant? How can you hotswap when the units are back to back? Do they have a fiberchannel interface? Can you do anything other networking than serial/Gig-E? etc. It seems like not because they cut the length of the rack in half.
All in all, Utah is basically ordering a 1000 node BladeCenter from a company with no previous blade experience (they're ultra dense systems were simply two dual Athlon mobos crammed into a 1U case). I thought that nobody would take on a strategy as risky as VATech (building a cluster from an unreleased computer), but I was wrong.
If they were both finished today, I think the Utah system would have a higher Rpeak (I can't be sure since I'm too lazy to look up everything). Rmax would be lower due to bandwidth limitations of the blades. Even the former would be reversed if someone got a version of double precision Linpack (the test for the Top100) to work with Altivec optimizations (an 4x increase)--there is some notes about Altivec optimized single precision, but none about double. Is that a limitation of the G4 Altivec and does this limitation carry over to the G5 VMX? Anyone?
Take care,
terry
Thanks!
I guess this answers that debate. It's $5.2 million for the systems itself as some of us guessed. The 5-year cost must have been a misquote.
I assume this answers my RAM reliability problem. BTW, I would think PSC means Pittsburgh Supercomputer Center, which makes a lot of sense. They were on par with SDSC (where the director of VaTech's project is from).
That answers all the people who were so sure it had to be Black Lab Linux. It turned out, as expected, it isn't an issue of kernel support of clustering since they're running an application-layer massively parallel system (MPI).
IBM xlc and xlf for the G5 were just released in Beta. Coincidence?
Should help with some of the costs...
Ouch, I was wrong about this. This still seems very aggressive to me but it does explain why IBM entered the bid with an Opteron-based system (IBM G5 blades/servers won't be available until 1Q 2004) as well as answers the "why not an G5 XServe" rants.
This should make a certain someone happy. Note the reference to 24/7/365!
Can anyone explain what the heck this means?