New Apple Super Computer (Army)

[MacRumors]

Based on the success of Virginia Tech's PowerMac G5 supercomputer, CNet reports (http://news.com.com/Apple+sells+supercomputer+sequel/2100-1010_3-5242487.html?tag=nefd.top) that Colsa has placed an order with Apple for a number of servers:

A U.S. Army contractor has purchased a $5.8 million, 1,566-server supercomputer from Apple Computer, a real-world cousin to an academic system that briefly appeared high on a list of the most powerful machines.

The new cluster is expected to reach 15 teraflops when it is completed this fall and will run Army simulations of the aerodynamics of flight.

[Bernd]

Good news Apple

[impierced]

Based on the success of Virginia Tech's PowerMac G5 supercomputer, CNet reports (http://news.com.com/Apple+sells+supercomputer+sequel/2100-1010_3-5242487.html?tag=nefd.top) that Colsa has placed an order with Apple for a number of servers:



The new cluster is expected to reach 15 teraflops when it is completed this fall and will run Army simulations of the aerodynamics of flight.

This was rumored a while back from some foreign web site and everyone dismissed it. Anyone remember the link?

[geerlingguy]

Woohoo! Another one chalked up for Apple... how long will it be before we all have 50 Xserves in our basements? ;)

[Freg3000]

MacCentral says it will be more than 15 TF.

The supercomputer, named MACH 5, is expected to deliver peak performance capability of more than 25 TFlops/second. In comparison, the Virginia Tech supercomputer announced last year attained sustained performance of approximately 10 TFlops/second, according to Apple director of product management, server hardware, Alex Grossman.

Either way.....AWESOME.

I know twice as many xServes would scale directly to twice as much power, but just imagine if the 5.8 million dollars they used to buy all of these xServes was 58 million. Imagine Macs comprising the fastest computer in the world......still many times less than the 350 million dollar Earth Simulator!

:D :D :D

[macridah]

25 teraflops ... that's blazing crazy fast. Go Army.

[Freg3000]

This was rumored a while back from some foreign web site and everyone dismissed it. Anyone remember the link?

Well this isn't foreign, it is just AppleInsider (http://www.appleinsider.com/news.php?id=355) :

Meanwhile, TechWeb is reporting that Virginia Tech has received "a number of inquiries" from federal agencies to use the university's installation or its supercomputer-kit technology to build their own supercomputer installations, following the universities announced plans to transition the cluster to Apple's new dual Power PC 970 Xserve G5 systems.

Argonne National Lab, the National Security Agency, and NASA, are among those expressing interest in the supercomputer technology. According to the article, negotiations were underway with potential customers who could use the university's installation itself, or obtain rights to build their own supercomputer based on the university's technology.

I do remember another source though.

[musicpyrite]

What?!!?!? WTF?? The government actually doing something right?

Good news for Apple. Good news for the Army. Bad news fo' M$ and DuLL.

[nuclearwinter]

WTF?!?!?!?!?!?! This is totally sweet! I am flipping out!

[impierced]

Well this isn't foreign, it is just AppleInsider (http://www.appleinsider.com/news.php?id=355) :



I do remember another source though.

I remember a site (maybe it wasn't foreign) that listed the reasons as to why the XServe G5 (or was it the PowerMac) wasn't shipping. One was because the US Government (I think) was purchasing 1500+ systems and another reason was something that was going to get someone at Apple in hot water. There were like 3 or 4 reasons.

ARGH, stupid memory.

[deepkid]

Based on the success of Virginia Tech's PowerMac G5 supercomputer, CNet reports (http://news.com.com/Apple+sells+supercomputer+sequel/2100-1010_3-5242487.html?tag=nefd.top) that Colsa has placed an order with Apple for a number of servers:

The new cluster is expected to reach 15 teraflops when it is completed this fall and will run Army simulations of the aerodynamics of flight.

I wonder if they'll run into any bottlenecks using gigabit ethernet instead of InfiniBand. Would that affect them truly reaching 15 teraflops...

[nsb3000]

Based on the success of Virginia Tech's PowerMac G5 supercomputer, CNet reports (http://news.com.com/Apple+sells+supercomputer+sequel/2100-1010_3-5242487.html?tag=nefd.top) that Colsa has placed an order with Apple for a number of servers:



The new cluster is expected to reach 15 teraflops when it is completed this fall and will run Army simulations of the aerodynamics of flight.

In related news, the new list of the top 500 super computers (http://www.top500.org/list/2004/06/) just came out, and I can't find Virginia Tech's Powermac G5 cluster anywhere on it. Did it really slide from #3 all the way to 501 and beyond?

EDIT: I answered my own question. The article states: "System X, which vanished from the most recent list for upgrades,"...

[coolfactor]

I wonder if they'll run into any bottlenecks using gigabit ethernet instead of InfiniBand. Would that affect them truly reaching 15 teraflops...

If you read the article, it mentions that they don't need the network bandwidth that the Big Mac needed. We'll see.

http://news.com.com/Apple+sells+supercomputer+sequel/2100-1010_3-5242487.html?part=rss&tag=5242487&subj=news.1010.5

[Sunrunner]

[QUOTE=coolfactor]If you read the article, it mentions that they don't need the network bandwidth that the Big Mac needed. We'll see.

I wonder why that would be ... isn't network bandwidth one of the primary factors that kill the speed of cllustered computers?

[coolfactor]

In related news, the new list the top 500 super computers (http://www.top500.org/list/2004/06/) just came out, and I can't find Virginia Tech's Powermac G5 cluster anywhere on it. Did it really slide from #3 all the way to 501 and beyond?

EDIT: I answered my own question. The article states: "System X, which vanished from the most recent list for upgrades,"...

It's not on there because they're in transition to an Xserve-based setup. Apple sure must've taken a hit for that, given the delays of delivering the new machines.

[coolfactor]

I wonder why that would be ... isn't network bandwidth one of the primary factors that kill the speed of cllustered computers?

It could be the way the processing is done. There would be a difference between transferring digital media files and transferring mathematical data, though.

[må¥å]

With this news and the news previously about the USarmy icons for the login this is looking good for apple.

And I remember people saying that Apple is going to go out of business in another 20 years. ;)

[mactarkus]

As someone who works for the Department of Defense, this purchase of Apple computers is monumental toward more widespread adoption of the platform. You see, the US Government = Microsoft. Even Linux is still a hard sell most of the time. I remember there was a big purchase by the US Navy a year or so back to outfit a submarine. I wonder how that project is progressing.

[nsb3000]

It's not on there because they're in transition to an Xserve-based setup. Apple sure must've taken a hit for that, given the delays of delivering the new machines.

Its true. Apple really needs to get over their "we make great products but have no idea how to produce them in volume" problem...

[AidenShaw]

[QUOTE=coolfactor]If you read the article, it mentions that they don't need the network bandwidth that the Big Mac needed. We'll see.

I wonder why that would be ... isn't network bandwidth one of the primary factors that kill the speed of cllustered computers?

It depends on the application.

The important factor is the ratio of the amount of data needed per "work unit" and the amount of CPU time needed to process the "work unit".

By far one of the most massive supercomputers around is the one doing SETI@home - and it's running on dialup, DSL and cable modems. The key is that it needs many CPU hours of work on a data packet of a few hundred KB.

For MPI jobs like LINPACK, however, you need to work on a few KB of data for a fraction of a second.

A slow or high-latency network will kill MPI performance. It won't matter for video rendering or other tasks with the right profile.

[Exponent]

Cool Beans. Go USArmy!

May the calculations of this computer help protect our awesome troops around the world.

[macsrus]

[QUOTE=coolfactor]If you read the article, it mentions that they don't need the network bandwidth that the Big Mac needed. We'll see.

I wonder why that would be ... isn't network bandwidth one of the primary factors that kill the speed of cllustered computers?

As a general rule faster interconnects=faster clusters....
But this is very Problem/Code related....

If your particular code does alot of message passing then a low latency interconnect such as Quadrics, Myrinet, or Infiniband is the way to go....

But some Problems are so CPU intensive... that they are literally CPU bound and for these Codes low latency interconnects are not as important...
And you can get by with Ethernet

Since they say they dont need a high speed interconnect, I would expect that their code is like the later

[deepkid]

It could be the way the processing is done. There would be a difference between transferring digital media files and transferring mathematical data, though.

Would seem like flight simulations would need speedy networking also. I guess we'll see once they get it up running. BTW, I did read that statement about networking needs in the article, but it still left me wondering.

[sushi]

As someone who works for the Department of Defense, this purchase of Apple computers is monumental toward more widespread adoption of the platform. You see, the US Government = Microsoft. Even Linux is still a hard sell most of the time.
Well put!

Hey, the government/DoD is not anti Mac and pro Microsoft. You could order any computer as long as it ran Windows natively and had Microsoft Office installed! :D

I really enjoyed it when the Army switched over from NT to Mac servers. It was so fun to see the look on the IT folks faces when I informed them of the change. Total disbelief.

...Of course I gladly enlightened them! :D :D :D


I remember there was a big purchase by the US Navy a year or so back to outfit a submarine. I wonder how that project is progressing.

Didn't hear about this. Good news!

I remember the ship (destroyer I believe) that was run on WinNT that totally crashed and they had to tow it back into the yard to fix it. Funny!

Sushi

[mactastic]

As an added bonus, the Army will probably spend a good deal less worrying about software security issues.

It's good to see people who make decisions based on the bottom line choosing Apples again.

(And let's keep this thread out of the political section.)

[QCassidy352]

can we please keep this computer-focused?

We all have an opinion, good or bad, of the US Army, but the point is that this is great for apple. First, because it's a 5.8 million dollar order. But second, because people are apparently starting to realize the power-per-dollar that you get with an x-serve cluster. Hopefully, this will just be the second of many such clusters.

(p.s. how about they get VT's Big Mac back on line???)

[AstroPlain]

> global thermonuclear war||

Do you think they could add a 300 baud modem to one of them for me, for unspecified reasons? :D

[Les Kern]

As an added bonus, the Army will probably spend a good deal less worrying about software security issues.

Nah, probably a closed system anyway.

[Wonder Boy]

awesome. an army of 1566.

[nagromme]

Come November, I wonder how many Top 5 supercomputers will be running Mac OS X. I don't see a date for MACH 5, but Big Mac should be back long before then. 2 of the top 5 seems likely then--maybe more!

And why do I get the feeling that the waiting list for Xserves is going to remain in the realm of iPod Minis?

[lefty111]

"Be all that you can be..."

[law guy]

Yes, well, unfortunately this is an inherently political matter.

However, I will save my comments for a political forum.

Politics or policy of the New American Century aside - it's not the army who's directly purchased these - i.e., the Secretary of the Army has not all of a sudden gone Mac happy. It's a military contractor that won the work set out in an solicitation or has been approved for on-going work, and was in turn able to purchase a system with some amount of their contract award. Think of it like this - you wouldn't call Boeing or Lockheed Martin the Airforce, would you? Same sort of deal.

[Telomar]

For reference the submarine order was for the sonar systems on US submarines.

Would seem like flight simulations would need speedy networking also. I guess we'll see once they get it up running. BTW, I did read that statement about networking needs in the article, but it still left me wondering.CFD packages are very computationally limited and lead themselves to clustering very well. It won't do as well on the Top500 list but the army isn't buying the computer just for that.

[ifjake]

cool, apple. but what it seems to me is that apple is becoming a inexpensive provider of computing power, which seems to be just a larger scale of that niche market thing. not all businesses need supercomputers. when apple computers start being used to do mere daily office work, that's where the growth will come from. but the larger the company the more expensive and difficult it is to make that switch. i could see a small startup company that begins with macs and grows with macs as being the most probable situation. i am personally happy with apple's performance. they make an awesome personal home computer and are always finding ways to innovate. i would be happy with just that, but i guess we'll see how things go.

heh, fine no politics. :rolleyes:

[narco]

I don't get it -- what's the point of having one of the fastest computers? Bragging rights? What do they do on these computers, Photoshop tests?

// narco

[windowsblowsass]

I don't get it -- what's the point of having one of the fastest computers? Bragging rights? What do they do on these computers, Photoshop tests?

// narco

in this case probably stuff that were not allowed to know about ;)

[bnemesis]

Will the system admin's login for MACH-5 be "speedracer"??? :p

[iLilana]

what rating or rank is virginia tech cluster now?

[quagmire]

First to note, that Virginia tech is not on the list because they do not have a supercomputer running so they had to drop out.
With the rumored 2.3 Ghz G5 Xserve update couldn't we see alot more speed out of them. Apple doesn't tell us the speed and I think everyone is presuming 15 Teraflops is with the 2.0 Ghz dual G5 Xserve. How do you calculate the speed of a super computer made up by 1500+ running dual 2.3 Ghz G5. With the itanuim supercomputer running at 20 tf, and we reach 25 tf hello 2nd.

[macmumma]

ABOUT TIME THE ARMY GOT SERIOUS

[tortoise]

I wonder if they'll run into any bottlenecks using gigabit ethernet instead of InfiniBand. Would that affect them truly reaching 15 teraflops...


Interconnect latency matters a lot. The "teraflop" benchmark is dependent on some very creative definitions with a very long list of conditionals. Only a very small number of possible applications will see anything remotely close to that performance with Infiniband, and even fewer with GigE. The codes they will be running are reasonably well suited for the PPC architecture though.

Differences in interconnect latency determine how an application will scale across a cluster. Some applications are more dependent than others, and identical machines with different interconnects can show a huge difference in performance even if the interconnects have roughly the same bandwidth. Clusters with very low latency interconnects are capable of running far more applications reasonably well than clusters with high latency interconnects. The Top500 benchmark is not terribly dependent on interconnect performance, even though most supercomputing applications are.

[Longey Nowze]

... I remember there was a big purchase by the US Navy a year or so back to outfit a submarine. I wonder how that project is progressing.

IIRC the the Navy bought G4 XServes that ran Yellow Dog Linux, I forgot the name of the company that sold Apple hardware running YDL, if you go to the site I'm sure you will find info on it

[tortoise]

I wonder why that would be ... isn't network bandwidth one of the primary factors that kill the speed of cllustered computers?


It is usually the interconnect latency rather than the bandwidth that kills clustered computers. Cluster interconnects like Myrinet and Infiniband don't have any more bandwidth than various flavors of Ethernet. What you are paying for is an order of magnitude reduction in latency versus Ethernet. That is what makes it worth the price, not the bandwidth.

And the folks that buy huge ccNUMA systems get an additional order of magnitude reduction in latency. Each step up in latency reduction costs a significant premium.

[macsrus]

Will the system admin's login for MACH-5 be "speedracer"??? :p

I believe Speed Racers car was the MARK V not MACH 5

[virividox]

SWEEEET, hopefully more people realize how powerful macs really are

[bnemesis]

I believe Speed Racers car was the MARK V not MACH 5

I had to go check. Its definatly MACH 5.
http://www.speedracer.com/cars-mach5.htm

...unless its only the mach 5 in the States? :confused:

[macsrus]

First to note, that Virginia tech is not on the list because they do not have a supercomputer running so they had to drop out.
With the rumored 2.3 Ghz G5 Xserve update couldn't we see alot more speed out of them. Apple doesn't tell us the speed and I think everyone is presuming 15 Teraflops is with the 2.0 Ghz dual G5 Xserve. How do you calculate the speed of a super computer made up by 1500+ running dual 2.3 Ghz G5. With the itanuim supercomputer running at 20 tf, and we reach 25 tf hello 2nd.

The theoretical performance of a PPC970 is

2 Floating point units X 2 flops per clock cycle x the GHZ of the CPU

Therefore 1566 x 2 cpus ea X 8 GigaFlops = 25.06 TF Theoretical

SO I would assume they are using the 2.0s

Now they claim they will get about 15 TF actual

15/25= .60

So they are assuming that they can reach about 60% efficientcy on Linpack

That is fairly high percentage considering the use of Gigabit Ethernet, but it could be attainable with some good performance tuning.....

1st it would depend on how much memory the nodes have.... the more the better.... because a larger LINPACK problem can be run which will cause the system to spend more time calculating than sending messages...

2nd. The compiler version makes a huge difference......If it was my cluster I would use IBM's xlf compiler.

3rd. The BLAS libraries you use are very important.... I saw the other day that GOTO has just finished a set of customed tuned BLAS libraries for the PPC.... If it was my system I would use them cause GOTO writes the best.

4th. Even though they selected Gig E ..... all Gig E isnt created equal.....
Some are much better than others.... and suprisingly some have fairly low latency......Example Extreme's Blackdiamond is even as low as 10 microseconds....
Now they didnt say whose Gig E they used.... But if they went with Extreme that could help.

5th The MPI you use maters a good bit too..... 3 real choices here....
MPICH.... free
LAM..... free
MPI PRO.... expensive

While mpipro is the best overall
LAM is very quick on TCP/IP
Mpich is more like the middle of the road and kind of the standard in research systems.

6th and Final.... Linpack has a lot of tuning parameters.... If these guys are really good they can squeak some extra FLOPS out of tunning linpack.

So who knows.... maybe they will get 15 TFlops.... heck maybe they will get more.. maybe less....
but at least its another MACINTOSH and a win for APPLE!!!!

[macsrus]

I had to go check. Its definatly MACH 5.
http://www.speedracer.com/cars-mach5.htm

...unless its only the mach 5 in the States? :confused:

I stand corrected.....

[thogs_cave]

As someone who works for the Department of Defense, this purchase of Apple computers is monumental toward more widespread adoption of the platform. You see, the US Government = Microsoft. Even Linux is still a hard sell most of the time. I remember there was a big purchase by the US Navy a year or so back to outfit a submarine. I wonder how that project is progressing.

Back in the late 80's/early 90's there was the Government's DeskTop-III (I think it was DT-III) contract, and it was SCO UNIX on X86 (Unisys, IIRC). I remember it because I used to work for an integrated sofware company (Enable) that actually did a port of their popular-with-the-feds DOS software to SCO/X Windows. So, there was a time when the Gov't did something other than MS. Perhaps we will see more and more coolness in the hands of our public servants....

Just imagine the IRS on Macs. Refunds sent out the next day! :eek:

[iomar]

Well, I hope apple remembers us when they are making these back deals. It is because of us that apple is where it is.

[Rower_CPU]

Any political commentary is off-topic for this discussion and will be immediately deleted. Please do not continue to post these types of comments.

Thank you :)

[adamjay]

so its jerks with computers like these that are smoking the rest of us at UT2004.

[ErikGrim]

so its jerks with computers like these that are smoking the rest of us at UT2004.Obviously you mean "smoking the rest of us in America's Army."

[Analog Kid]

This certainly gives Apple some credibility. I hope the ethernet doesn't hurt performance... There's precious few Mac clusters out there. It would suck if people started blaming Apple when the problem is with the design...

[brianrubinton@m]

I hope larger orders are sent to apple for supercomputers so that way we can see how a $350,000,000 mac supercomputer goes against the #1 supercomputer.

[mactarkus]

That's true. The Army did not directly purchase them. More than likely however, the Army did a source selection and selected this contractor, who had proposed this plan or, worked closely with the Government after contract award to select the systems. Either way, I'm sure the Army had final approval of the direction. Selecting the systems for such a huge computing project would have been a very big decision -- for the future of the project and politically. The Contractor might have brought the Apple option to the table, but the Army bought in.

Politics or policy of the New American Century aside - it's not the army who's directly purchased these - i.e., the Secretary of the Army has not all of a sudden gone Mac happy. It's a military contractor that won the work set out in an solicitation or has been approved for on-going work, and was in turn able to purchase a system with some amount of their contract award. Think of it like this - you wouldn't call Boeing or Lockheed Martin the Airforce, would you? Same sort of deal.

[macsrus]

This certainly gives Apple some credibility. I hope the ethernet doesn't hurt performance... There's precious few Mac clusters out there. It would suck if people started blaming Apple when the problem is with the design...

There isn't anything wrong with their design.....
In the press release they stated that their code/problems they are running didnt need a high speed(read low latency interconnect such as infiniband).

Now... how would they have know that.... Im sure they tested and know the particular requirments of their code.

If they would have bought infiniband, myrinet, or quadrics and didnt need it.... then that would be a poor design....
Especially when any of those 3 interconnects can almost double the price of the cluster

Computer systems are tools used to solve problems.......

It sounds to me like the Colsa guys did their homework and built a great system/tool specifically designed to solve their problems...

So if the system doesnt rate as high on the TOP500 as it could have with a high speed/low latency interconnect... Im sure their customer doesnt really care.

[oingoboingo]

Its true. Apple really needs to get over their "we make great products but have no idea how to produce them in volume" problem...

Does Apple even produce anything themselves anymore? All the iBooks and PowerMacs are made by Taiwanese or Chinese contractors. Same with the iMacs. I also believe the PowerMac G5s are made under contract. Pretty sure the iPods are a contract job too. Aren't the Xserve delays more likely to be due to IBM?

[nsb3000]

Well put!

Hey, the government/DoD is not anti Mac and pro Microsoft. You could order any computer as long as it ran Windows natively and had Microsoft Office installed! :D

I really enjoyed it when the Army switched over from NT to Mac servers. It was so fun to see the look on the IT folks faces when I informed them of the change. Total disbelief.

...Of course I gladly enlightened them! :D :D :D



Yup, it is good to see that Apple is finally getting some respect. I agree with other readers, that this is *big* news, because if the Army is saying this is worth while, then others will fallow.

[machan]

Does Apple even produce anything themselves anymore? All the iBooks and PowerMacs are made by Taiwanese or Chinese contractors. Same with the iMacs. I also believe the PowerMac G5s are made under contract. Pretty sure the iPods are a contract job too. Aren't the Xserve delays more likely to be due to IBM?

This is exactly why Apple needs to start ordering more of all of the above from the contractors to keep up with demand. It's nice that there's a new supercomputer on the block, but it shouldn't be at the expense of making regular customers wait 6 months for their Xserve to arrive.

[pkradd]

Technically, the U.S. Army is not purchasing the X-Serves. They have a contract with an outside vendor, in this case COLA, to supply them with a super computer. COLA made the decision to purchase the X-Serves which they will provide to the Army and set up. For COLA it probably meant a higher profit because of the excellent price to performance ratio. The VA Tech X-Serves are still being delivered and they won't be on line for another month or so. Go Apple.

[sushi]

That's true. The Army did not directly purchase them. More than likely however, the Army did a source selection and selected this contractor, who had proposed this plan or, worked closely with the Government after contract award to select the systems. Either way, I'm sure the Army had final approval of the direction. Selecting the systems for such a huge computing project would have been a very big decision -- for the future of the project and politically. The Contractor might have brought the Apple option to the table, but the Army bought in.
Some contracts can be so specific as to specify the hardware and software used.

In the past, there were many DoD folks who were anti-Mac and did everything that they could think about to stifle the procurement and use of Macs.

At one location I was at, they went so far as to eliminate any non-Microsoft software. As the IT for our department, I battled the higher headquarters IT folks to keep using Harvard Graphics, which as that time was loads better for what we needed to do. Fortunately, we had original copies for each individual computer so they couldn't say much when I brought up that fact. Then they said that they would not provide any support for HG. I said fine. We don't need it.

Can you say a combatative environment?!

I am so happy to see Macs get back in the government workplace, albiet slowly.

Sushi

[sinisterdesign]

man, this is going to make a really SWEET Halo server...

[jettredmont]

Woohoo! Another one chalked up for Apple... how long will it be before we all have 50 Xserves in our basements? ;)

About as long as it takes me to scrounge up a hundred grand or so ...

... and a basement ...

[socbyset]

now, the next task is to get these guys to run folding@home for the macrumors team when they're not running any fluid dynamics simulations...

[The Red Wolf]

Does Apple even produce anything themselves anymore? All the iBooks and PowerMacs are made by Taiwanese or Chinese contractors. Same with the iMacs. I also believe the PowerMac G5s are made under contract. Pretty sure the iPods are a contract job too. Aren't the Xserve delays more likely to be due to IBM?

Up until a few months ago when the Elk Grove CA manufacturing facility was closed, PowerMac G5s were produced just south of Sacramento. They're now produced in conjunction with a California based independent contractor, within California.

[greg75]

I remember the ship (destroyer I believe) that was run on WinNT that totally crashed and they had to tow it back into the yard to fix it. Funny!
They should have used MacOS X instead, because as well all know it's impossible for applications to crash when running under MacOS X.

[omala]

There are a lot of factors in parallel supercomputer performance, and what's "faster" is somewhat application specific. When I worked with Crays a few years ago, for example, they used relatively slow CPUs for the day - but things like memory and disk access (and message passing between nodes) were tuned so that if you wrote your code properly, you could stream data from node to node, or (on a vector machine) from anywhere in main memory with ultra-low latency.

As for scaling performance as you increase the number of CPUs (more relevantly called "nodes" in this case), it depends on what you're doing. Some problems are what are called "embarassingly parallel", which is to say you can completely divide them up into a huge number of chunks that can run completely independently of each other - those problems really will scale almost linearly as you add processors (with a given number of "control nodes" to feed problems to the rest). With its divide-and-conquer approach, SETI@home actually comes close to this extreme. Other problems require all of the results from previous steps to feed the data for the next step, and are really run serially no matter how many nodes are available.

In real life, most interesting problems fall somewhere in the middle, but when you're designing a massively parallel system you have an idea of which way you're trying to tune it. I'm pretty sure this system's been modeled thoroughly before someone signed the deal. ;)

[shewy]

Sell Sell Sell!! Send my stock up! It'll be interesting to see how the Apple marketing people handle this one - involving the military and all...

[Apple Hobo]

I don't get it -- what's the point of having one of the fastest computers? Bragging rights? What do they do on these computers, Photoshop tests?

// narco

There's A LOT more to supercomputing than screwing with Photoshop. Supercomputers don't even run Photoshop.

Here's an example of what supercomputers do:

In conjunction with some of the world's most creative scientific and engineering minds, these formidable tools already have made automobiles safer and more fuel-efficient; located new deposits of oil and gas; saved lives and property by predicting severe storms; created new materials and life-saving drugs; powered advances in electronics and visualization; safeguarded national security; and unraveled mysteries ranging from protein-folding mechanisms to the shape of the universe.

You can read more here: http://cray.com/industry/

[dontmatter]

how many computers were in VA tech's super? How far were they from the theoretical efficiency?

[morespce54]

There's A LOT more to supercomputing than screwing with Photoshop. Supercomputers don't even run Photoshop.



No... seriously???? ;) ;)
Too bad for Adobe then... :D

[AidenShaw]

how many computers were in VA tech's super? How far were they from the theoretical efficiency?

The theoretical peak of the Xserve is 16 GFLOPS. (2 CPUs * 2 GHz * 4 Ops/cycle)

VATech had 1100 PowerMac G5s (same peak), for a combined theoretical peak of 17,600 GFLOPs.

VATech measured 10,280 GFLOPs, or about 58% effeciency.

[jcdenton]

This is great news for Apple, although let's keep some perspective with the comparisons to other supercomputers on the Top 500. Yes, this is a jump over the "Big Mac," but the rest of the competition is not exactly standing still either.

And of course it will be bumped as the top PowerPC machine in a year or so when IBM's Blue Gene goes online at Livermore in California.

[shamino]

I remember the ship (destroyer I believe) that was run on WinNT that totally crashed and they had to tow it back into the yard to fix it. Funny!
That would be the USS Yorktown (http://www.wired.com/news/technology/0,1282,13987,00.html).

[shamino]

I don't get it -- what's the point of having one of the fastest computers? Bragging rights? What do they do on these computers, Photoshop tests?
In this particular case, according to the C|Net article (http://news.com.com/Apple+sells+supercomputer+sequel/2100-1010_3-5242487.html), it is being used for hypersonic aerodynamic simulation. That is, a simulation of how aircraft parts behave at speeds much faster than the speed of sound.

Probably as a part of designing new vehicles/weapons that move at hypersonic speeds.

[shamino]

Cluster interconnects like Myrinet and Infiniband don't have any more bandwidth than various flavors of Ethernet. What you are paying for is an order of magnitude reduction in latency versus Ethernet.
The latency issue is definitely what drives up the price. So is jitter (that is, per-packet variation in the delay characteristics - important for some applications, like video.)

But there is still a big bandwidth difference. According to the Infiniband (http://www.infinibandta.org/home) FAQ (http://www.infinibandta.org/specs/faq/), Infiniband's link speeds are 2.5, 10 and 30Gbps. (I also remember reading that 100G Infiniband is under development for the future.)

Ethernet, on the other hand, doesn't go that fast. The only commonly used speeds are 10M, 100M and 1G. There's a spec for 10G, but there are very few vendors shipping any devices at that speed.

[shamino]

4th. Even though they selected Gig E ..... all Gig E isnt created equal..... Some are much better than others.... and suprisingly some have fairly low latency......Example Extreme's Blackdiamond is even as low as 10 microseconds....
Now they didnt say whose Gig E they used.... But if they went with Extreme that could help.
I hope they don't plan on using the 1000BaseT interface built-in to the motherboard. :eek:

[xStep]

They should have used MacOS X instead, because as well all know it's impossible for applications to crash when running under MacOS X.

Sorry. Gotta call you on that one. That is a load of malarky!

[socbyset]

Sorry. Gotta call you on that one. That is a load of malarky!

I was assuming he was being sarcastic.. could be wrong.

[oingoboingo]

Up until a few months ago when the Elk Grove CA manufacturing facility was closed, PowerMac G5s were produced just south of Sacramento. They're now produced in conjunction with a California based independent contractor, within California.

That may have been only the case for PowerMac G5s bound for the US market. My 1.6GHz G5 (ordered in August 2003, arrived in October 2003) has a 'Made in China' notice on the system information panel located at the bottom of the case on the access panel side (I'm in Sydney).

[macsrus]

I hope they don't plan on using the 1000BaseT interface built-in to the motherboard. :eek:

Actually the Gig E that is built into the xserve G5 mother board is rather quick.....
It is vastly improved over the G5 desktop or the xserve G4....

And should perform better than any third party adapter that could be installed into the PCI-X bus... since the 2 Gig E chipsets on the G5 are not in the southbridge... and actually have there own pipes to the hypertransport of the CPU's and Memory.....

[tortoise]

But there is still a big bandwidth difference. According to the Infiniband (http://www.infinibandta.org/home) FAQ (http://www.infinibandta.org/specs/faq/), Infiniband's link speeds are 2.5, 10 and 30Gbps. (I also remember reading that 100G Infiniband is under development for the future.)

Ethernet, on the other hand, doesn't go that fast. The only commonly used speeds are 10M, 100M and 1G. There's a spec for 10G, but there are very few vendors shipping any devices at that speed.


There is as much Ethernet available as Infiniband at those respective bandwidth levels. 10G Ethernet can be had now, and can be driven at >100G using WDM, channel bonding, etc. That's the great thing about fiber Ethernet actually: there is no real limit to the maximum bandwidth you can buy if you want to spend some cash. To put it another way, if they can switch Infiniband that fast, then they can certainly switch Ethernet that fast with its looser constraints. The limit is the switch fabric. While you don't see local offices using 10G+ fiber Ethernet links, it is used by long-haul and metro fiber providers, particularly as Ethernet slowly becomes THE protocol standard for all backbone networking.

Different technologies for different purposes really. Infiniband, Myrinet, Quadrics, RDMA, etc are designed for low latency above all else, with bandwidth coming second. Ethernet is designed for extremely scalable and efficient bandwidth, but sacrifices average latency to do it. Ethernet, being as ad hoc as it is, can get closer to theoretical than anything else when it comes to real-world throughput.

For my own supercomputing needs, I've been eyeballing the Octiga Bay gear (recently acquired by Cray). That is one hell of a fabric. Terabit scale fabrics with 1-us latency across hundreds of processors. Damn... I would love to see a benchmark from one of those systems, especially with the sick amount of floating point performance the new Pathscale compilers can squeeze out of that already fast architecture.

[macsrus]

For my own supercomputing needs, I've been eyeballing the Octiga Bay gear (recently acquired by Cray). That is one hell of a fabric. Terabit scale fabrics with 1-us latency across hundreds of processors. Damn... I would love to see a benchmark from one of those systems, especially with the sick amount of floating point performance the new Pathscale compilers can squeeze out of that already fast architecture.

Yes the Octiga Bay stuff is really sweet.... It was designed using FPGA's.....and ASICS....
The only problem is a question of scalability....
While Octigabay claims "those low latencies are one reason why the machine can scale to a ridiculously large 12,000 processors in a single machine"
I talked to Cray about these and they were not as optimistic about its scalability.
CRAY said that they were only planning to offer it for smaller cluster sizes.....512 processors and below....
Using current Opterons that will only get you in the 3 to 4 TFlop range for quite a healty price tag....

[shamino]

There is as much Ethernet available as Infiniband at those respective bandwidth levels. 10G Ethernet can be had now, and can be driven at >100G using WDM, channel bonding, etc.
I'm going to have to call out your error here.

WDM and channel bonding are not Ethernet technologies. They're optical technologies that you can choose to layer Ethernet over. You can layer lots of other things over it as well.

If you use this as an argument for saying that Ethernet currently runs faster than 10G, then I can use that same argument to claim that V.90 dial-up runs faster than 10G. After all, I can use SONET muxes to merge millions of voice channels into a single OC-192 and then use WDM to mux dozens of those onto a single fiber.
That's the great thing about fiber Ethernet actually: there is no real limit to the maximum bandwidth you can buy if you want to spend some cash.
No. That's the great thing about fiber, period. I can do the same WDM and channel bonding with ATM, Frame Relay, POS, or any other fiber-based technology and hit those same bandwidth levels.

Your argument says nothing that's unique or special to Ethernet.

As for what service providers are using for their metro-area facilities, you know as well as I do that there is no universal consensus over what is "best" here. There are a lot of competing technologies and different providers are using different systems. Price and politics play as much a role in selection as the technical merits of any given system.

[sushi]

That would be the USS Yorktown (http://www.wired.com/news/technology/0,1282,13987,00.html).
Thanks. Good article.

Sushi

[tortoise]

I'm going to have to call out your error here.

WDM and channel bonding are not Ethernet technologies. They're optical technologies that you can choose to layer Ethernet over. You can layer lots of other things over it as well.


No, you are somewhat incorrect here. First, channel-bonding is not an optical technology. While in theory you could channel-bond most Layer-2 network protocols, in practice you can't. Every Ethernet fabric that is worth a damn, right up to the 10G terabit fabrics, has bonding built into it. It could be copper, fiber, wireless, WDM, or whatever, as long as it is an Ethernet fabric, you can bond arbitrary channels in the fabric. Most channel bonding is done as a Layer-3 protocol hack.

The WDM is convenient primarily because it means you don't have to run a physical layer to each port while still using the bonding capabilities of the fabric. That's more of a convenience issue.


If you use this as an argument for saying that Ethernet currently runs faster than 10G, then I can use that same argument to claim that V.90 dial-up runs faster than 10G. After all, I can use SONET muxes to merge millions of voice channels into a single OC-192 and then use WDM to mux dozens of those onto a single fiber.
No. That's the great thing about fiber, period. I can do the same WDM and channel bonding with ATM, Frame Relay, POS, or any other fiber-based technology and hit those same bandwidth levels.


Again, not really true. With the Ethernet fabrics there is no translation layer, but with most of the other things you mention there is a pretty big bottleneck in actuality with the encapsulation and layer translation to do all that. A properly done high-performance Ethernet fabric is completely "flat" from edge to edge. To put it another way, Ethernet fabrics are two parallel highways that both go to the same endpoint, and they can choose to take either at the fork. For those other protocols, when they get to the fork for the parallel highways, they are forced to change vehicles.

The bandwidth of a given Ethernet network is the same as the bandwidth of its switch fabric. While you can bond other protocols, the consequences and broader interactions are a little different. (Not that anyone on this thread really cares about those details.)


As for what service providers are using for their metro-area facilities, you know as well as I do that there is no universal consensus over what is "best" here. There are a lot of competing technologies and different providers are using different systems. Price and politics play as much a role in selection as the technical merits of any given system.


Actually, a pervasive Ethernet Layer-2 is slowly becoming the concensus technology. There are a lot of reasons why, but primarily it is because it is superior to SONET, ATM, etc by just about any metric you care to use (cost, latency, bandwidth, reliability). There are already companies that use a globally switched Layer-2 Ethernet network from edge to edge. Ethernet fabrics are ridiculously fast and extremely flexible if you don't jump into Layer-3.

[macsrus]

I just heard that Colsa plans to triple the size of the Armys cluster by years end. It is suppose to grow to over 4672 xserves.

Can You imagine thats over 74 TFlops

I also heard that they will be adding Infiniband to the complete system later this year.....

Cool News Huh......

[iMook]

Even if the Army supercluster were scaled to 74 TFlops, it still wouldn't compare at all to a full-scale IBM Blue Gene/L run. Projected power: 340 TFlops (next year).

Early on, someone mentioned a $58 million cluster. Even that, assuming linear speed scaling, wouldn't top the Blue Gene/L. Sorry, but IBM's got Apple beat on this one.


P.S. - A 1/16-scale Blue Gene/L prototype landed #4 on the Supercomputer List. Now that's power. Of course, you COULD argue that it runs off of PowerPC processors. :p

[macsrus]

Even if the Army supercluster were scaled to 74 TFlops, it still wouldn't compare at all to a full-scale IBM Blue Gene/L run. Projected power: 340 TFlops (next year).

Early on, someone mentioned a $58 million cluster. Even that, assuming linear speed scaling, wouldn't top the Blue Gene/L. Sorry, but IBM's got Apple beat on this one.


P.S. - A 1/16-scale Blue Gene/L prototype landed #4 on the Supercomputer List. Now that's power. Of course, you COULD argue that it runs off of PowerPC processors. :p

Yes and you can also argue that the cost of Blue Gene/L will be over 350 million
noone will be running BG/L exclusively(i.e. as a single user running a single job).... It will more than likey be running hundreds of small jobs at the same time, ea using maybe a few hundred processors ea.
One could argue.... wouldnt it be better to buy 70 15 Tflop systems for the same money ?

[iris_failsafe]

Blue Gene will a couple of generations ahead of any scalar computer our there. The memory is attched to the processor.

[Les Kern]

....assuming linear speed scaling, wouldn't top the Blue Gene/L. Sorry, but IBM's got Apple beat on this one.


Good. I wish they'd just quit. I ordered my G5 deck-out tower 2 weeks ago, and I want it before it becomes a liquid-cooled pod in a Matrix-like college basement. :)

[jhu]

In this particular case, according to the C|Net article (http://news.com.com/Apple+sells+supercomputer+sequel/2100-1010_3-5242487.html), it is being used for hypersonic aerodynamic simulation. That is, a simulation of how aircraft parts behave at speeds much faster than the speed of sound.

Probably as a part of designing new vehicles/weapons that move at hypersonic speeds.

wouldn't it make more sense for the air force to acquire something with this much computing power?

[shamino]

In this particular case, according to the C|Net article, it is being used for hypersonic aerodynamic simulation. That is, a simulation of how aircraft parts behave at speeds much faster than the speed of sound.

Probably as a part of designing new vehicles/weapons that move at hypersonic speeds.
wouldn't it make more sense for the air force to acquire something with this much computing power?
What makes you think the Air Force doesn't employ contractors with supercomputer clusters already?

As for what the Army is doing with it, they're obviously not designing planes. The Army doesn't fly them. And a hypersonic helicopter sounds pretty ridiculous to me.

But there are other things that might be appropriate. Like hypersonic bullets (perhaps as fired from a rail-gun) or missiles.

[macsrus]

I'm very glad to hear that Apple's technologies and products will help push the frontier of aeronautics. To get past the current stagnation in commercial aviation technologies, basic research like hypersonics and scramjet engine are needed. This kind of computing power provides the researchers with the capability to model and predict hypersonic aerodynamics (computational fluid dynamics) and match experimental data from flight test programs ($$$$$$$$) to theoretical data. This in turn leads to better tools that allows more accurate predictions and more optimized designs for the next generation vehicles. Imagine vehicles that can carry passengers from LA to Tokyo in about 1 hour.

From the replies above, it's obvious that some people feel that Apple's contract for providing this research tool to the military is morally wrong. However, I must point out that the type of cost involved with aerospace research usually means that the private sector can not or will not make the long term R&D investment required. This is further exacerbated by the development of stock market based economy where quarterly results determines the success or failure of a company. Companies are increasingly unwilling to make the long term R&D commitments with corporate funding because the pay off may not come for 10 years or more. Take a look at the history of aviation...all the major advancements that became commercially viable operations are due to military investment in the necessary basic research and technology development. Jet engines and rocket engines are just two examples. Large aircraft and associated structural and materials technologies that make inexpensive air travel possible today are similarly due to military's needs for large transports and bombers.

Technology is always a two-edged sword. Any technology can be used to benefit the society or to hurt the society. It's not the technology or even the weapons that kills. It's the people that employ those weapons that has the ultimate moral responsibility.

It's good to know that our favorite fruit company is not only helping with bio-technologies, but also aerospace technologies. And if the rumors of CATIA being ported to OS X turns out to be true, that would be fantastic news. Companies like Boeing use thousands of CATIA workstations. The 777 was developed using CATIA running on IBM's workstations that were built on PowerPC's and AIX operating system. I would love to see some or all of these CATIA workstations in the aerospace industry replaced with OS X based CATIA systems. It would be nice to see Apple make a serious come back in the aerospace industry.

I couldnt have said this any better myself