Xgrid - Parallel and Distributed Computing

[MacRumors]

ThinkSecret.com discovered (http://www.thinksecret.com/news/xgrid.html) that Apple had set up a new mailing list for Xgrid users (http://www.lists.apple.com/mailman/listinfo/xgrid-users).

According to the description, "This mailing list is for user discussion use and deployment of Xgrid -- Apple's solution for parallel and distributed high performance computing."

'Xgrid' was originally trademarked by Apple (http://www.macrumors.com/pages/2002/11/20021102090635.shtml) in October 2002. There has been some subsequent rumors (http://www.macrumors.com/pages/2003/01/20030127223039.shtml) about it, but little direct information from Apple until now.

[mj_1903]

Very cool. I wonder when the release of this solution will be? Thinksecret seems to have no other information.

[Hawthorne]

Va. Tech, of course!
:D
Slashdot (http://apple.slashdot.org) has news that the Va. Tech supercomputer (nicknamed "Big Mac") is now the #2 dawg out there. Wow.

[cesar]

wwoowow. xgrid will be great!!!!

now, the only thing missing is the G5 Xserve....

[Pedro Estarque]

Can Photoshop take advantage of this?
:rolleyes:

[Raiden]

Wow, this sounds really cool!! I wonder when we will be able to take advantage of this...

[Catfish_Man]

Sweeeeeet.

[Mr.Hey]

G5 servers!!!

With the VT G5 supercomputer clocking at a theoretical 17.6 Tflops! this comes in at just about the right time.

http://apple.slashdot.org/article.pl?sid=03/10/12/143259&mode=thread&tid=107&tid=187&threshold=-1

[dongmin]

Things are falling into place, don't you think, for a World Wide Domination™ by Apple?

First OS X, then the G5, and now the Xgrid. VaTech was huge. And we've been hearing about Pixar potentially making a large investment in G5s.

[MrMacMan]

Woo that really kicks.

I mean great news for the G5 and massive Clusters everywhere.

Behind Earth Simulator which costed $350 million Dollars .

Thats AMAZING for Performance/Cost.

[Powerbook G5]

I wonder what Skynet would have been like if it were ran off of Apple instead of Dell...maybe we'll find out... ;)

[porky]

Call me stupid but, what's XGrid?

[gwangung]

Originally posted by porky
Call me stupid but, what's XGrid?

You're not stupid (but do you want me to call you that anyway?).

All that's happened is that Apple trademarked the term...and that's it. Everything that's known about the term is on this page.

[Catfish_Man]

Originally posted by gwangung
You're not stupid (but do you want me to call you that anyway?).

All that's happened is that Apple trademarked the term...and that's it. Everything that's known about the term is on this page.

No, what happened is that Apple set up a mailing list for a product called XGrid that's for clustering groups of computers. From other sources I've seen, it appears to be based off of Sun's Grid Engine.

[bobindashadows]

Doesn't this remind anyone of the XCode technology, which compiles across networks? Pretty nifty keen.

This is a random guess based on "parallel and distributed computing"...

[mvc]

Originally posted by Powerbook G5
I wonder what Skynet would have been like if it were ran off of Apple instead of Dell...maybe we'll find out... ;)

It would think different and just upgrade the human race instead of removing it:p

[cesar]

Originally posted by bobindashadows
Doesn't this remind anyone of the XCode technology, which compiles across networks? Pretty nifty keen.

This is a random guess based on "parallel and distributed computing"...

Shake for Macos X as a app called QMaster that renders across a network, searching for idle macs...

[TMay]

It is getting to the point that for the price of a compact car, we will be able to create a 5 node cluster of dual G5/dual G5 Xserve (assuming of course, that such Xserve exist).

I propose 5 nodes (10 processors) as a definition of a personal cluster.

Too ambitious?

[yamabushi]

Several groups have been experimenting with making small clusters for some time. Here is an old report of a 4 node G3 solution.
http://exodus.physics.ucla.edu/appleseed/archive/appleseed_report98a.html

I wonder how much performance you could get from a 5 node dual G5 cluster. Does anybody have more info on creating small clusters of Macs?

[ClimbingTheLog]

Odds are this is just PDO warmed over.

http://developer.apple.com/documentation/Cocoa/Conceptual/DistrObjects/index.html#//apple_ref/doc/uid/10000102i

NeXT has supported distributed computing for almost a decade.

That's good, of course, it's a mature technology with the bugs already worked out, i.e., ready to go. If they cold also support MPI that would be even better.

[Catfish_Man]

Originally posted by ClimbingTheLog
Odds are this is just PDO warmed over.

http://developer.apple.com/documentation/Cocoa/Conceptual/DistrObjects/index.html#//apple_ref/doc/uid/10000102i

NeXT has supported distributed computing for almost a decade.

That's good, of course, it's a mature technology with the bugs already worked out, i.e., ready to go. If they cold also support MPI that would be even better.

Doesn't POOCH support MPI?

[AidenShaw]

Originally posted by Hawthorne
...the Va. Tech supercomputer (nicknamed "Big Mac") is now the #2 dawg out there. Wow.

That's just quoting the theoretical performance - "being" means that you actually have to power on the computer and run some code.

There haven't been any statements that I've found about whether or not VTech actually was able to run the full benchmark suite by the 1 October deadline.

Also, note that the "theoretical #2" is 6 months out-of-date - who knows what the current list will be like. The "#2" six months ago might not be "#10" today....

[ClimbingTheLog]

Originally posted by Catfish_Man
Doesn't POOCH support MPI?

Yeah. Does XGrid incorporate POOCH? That would be neat.

[Catfish_Man]

Originally posted by ClimbingTheLog
Yeah. Does XGrid incorporate POOCH? That would be neat.

Not as far as I know, but POOCH runs on Macs just fine.

[alandail]

look at what a brand new dell based supercomputer costs

http://www.bizjournals.com/austin/stories/2003/09/29/daily37.html

$5.2 million for 17.6 TFlops with the G5
$38 million for 3 TFlops from Dell

Also, built in 3 months vs. a 5 year project. No wonder Va Tech went with Apple.

[AidenShaw]

Originally posted by alandail
$5.2 million for 17.6 TFlops with the G5

Have you seen the actual performance numbers for VTech, vs. the theoretical numbers that came from looking at the spec sheets and multiplying by 1100?

That 17 GFLOP number for VTech was calculated before the boxes were even delivered.

The "Top500" list, however, requires that you actually plug the computers into an electrical outlet and measure the performance. We don't know what percentage of "17" that number will be....

[Jeff Harrell]

Originally posted by cesar
Shake for Macos X as a app called QMaster that renders across a network, searching for idle macs... The same program comes with Compressor, which is bundled both with Final Cut Pro and DVD Studio Pro.

Anybody out there have a Shake 3 install they can refer to quickly? If so, please look at the file /etc/qmaster/QmasterApplications.plist. Is the SingleUser element set to true or false?

[Sherman]

Do you think VA Tech and Apple had an agreement about XGrid beforehand? It just seems like too amazing of a coincidence if you ask me. An awesome coincidence. VA Tech will definitely be running XGrid.

[dongmin]

Originally posted by alandail
look at what a brand new dell based supercomputer costs

http://www.bizjournals.com/austin/stories/2003/09/29/daily37.html

$5.2 million for 17.6 TFlops with the G5
$38 million for 3 TFlops from Dell

Also, built in 3 months vs. a 5 year project. No wonder Va Tech went with Apple.

The 3 TFlops represent their initial investment of 300 computers. The article states that they will add 200 more computers (for 2 TFlops) within a year. And the article is not clear on what exactly the 5-year period covers; it does not imply, however, that it took them 5 years to put 300 computers on line. And it doesn't also say what the cluster will look like after they've spent all of the $38 million. It could be a 20 TFlop for all we know.

For sure, the G5 cluster represents dramatic savings (in money and time) over previous clusters. We'll see how the value holds up over the next few years. I wouldn't be surprised to see a 2000-node dual 3 ghz cluster in a year. Can you say 40 TFlops for under $10 mil? The Earth Simulator is the current #1 doing about 40 TFlops at a cost of $350 mil. IBM is shooting for 150 TFlops with their Blue Planet which is slated for mid 2005.

[ipiloot]

Here:

http://don.cc.vt.edu/updatemeeting/slides/IMG_0647.JPG

one can see that the key person behind the VT Teracluster - Srinidhi Varadarajan wears possibly a Newton MP2x00 on he's waist.

[Pants]

Originally posted by ipiloot
Here:

http://don.cc.vt.edu/updatemeeting/slides/IMG_0647.JPG

one can see that the key person behind the VT Teracluster - Srinidhi Varadarajan wears possibly a Newton MP2x00 on he's waist.


uhh....its the 21st century and the man still has a *MULLET*!!

best keep him out of the PR shots... ;)

[LordMord]

All the St Martins Fashion students have been sporting mullets for a year nowas have the Paris fashion gurus...probably on their way out AGAIN...

I see your point though...:D

[mproud]

Originally posted by ipiloot
Here:

http://don.cc.vt.edu/updatemeeting/slides/IMG_0647.JPG

one can see that the key person behind the VT Teracluster - Srinidhi Varadarajan wears possibly a Newton MP2x00 on he's waist. And what's definitely a non-Apple computer laptop.

[AidenShaw]

Originally posted by Hawthorne
Slashdot (http://apple.slashdot.org) has news that the Va. Tech supercomputer (nicknamed "Big Mac") is now the #2 dawg out there. Wow.

The #2 computer on the list has an Rpeak of over 20 GFlops, more than the VTech cluster. Slashdot made the mistake of comparing the theorical performance of the Mac cluster against the measured performance of the computers on the list.

Of course, the list is sorted by measured maximum performance, not theoretical peak - so we don't know where the VTech will end up if it makes this fall's list....

[lmalave]

Originally posted by Sherman
Do you think VA Tech and Apple had an agreement about XGrid beforehand? It just seems like too amazing of a coincidence if you ask me. An awesome coincidence. VA Tech will definitely be running XGrid.

No, Virginia Tech wrote their own software called Deja Vu. Maybe Apple cut a special deal with Va. Teck so that they could get Va. Tech's code.

[D*I*S_Frontman]

What will running all 1100 VT Macs on the latest build of Panther do to the "real world" benchmarks if Panther is supposed to offer such huge speed increases to single G5 systems?

If XGrid turns out to be a fancy way to spread processing threads across several G5 systems (regardless of application), will we soon be entering the "supercomputing cluster for the rest of us" era? Or does every application have to be coded for multiple-system processing distribution?

It would be great to chew through high-def video editing with a rack of even a handful of DP G5 XServes. Time is money for audio engineering and video editing--@ $100/hr starting, how long does it take to justify spending $15K or so on such a system? Answer: less than 4 weeks @ 40hrs/week, or for those who do about 12 large scale projects per year, finishing each project two days ahead of schedule due to productivity increases. Cool.

[Powerbook G5]

So if this were the case, would we all someday be buying computers that are merely terminals for accessing a vast processing pool instead of how we do it today with picking one to fit our needs with their own CPU(s)? It would be interesting if 10 years from now we don't even have G4s, G5s, G6s in our computers, but simply fibre channels, wireless networks, and these massive distributed grids where we tap into all of the vast channels of processing power. Instead of saying "I have a dual 2 GHz G5, baby!" we could say "I have access to 6,000 quantum core G8s, baby!" Mmm...teraflops...

[tace]

Greetings! <p> I have been told by Oracle that next version of their Database software 10g will be available on OS X in Enterprise (vs. current Developer) format. By the way, "g" stands for Grid Computing. Coincidence? Think NOT!

[SilentPanda]

Originally posted by tace
Greetings! <p> I have been told by Oracle that next version of their Database software 10g will be available on OS X in Enterprise (vs. current Developer) format. By the way, "g" stands for Grid Computing. Coincidence? Think NOT!

And the roman numeral for 10 is... X!

10g = 10 + g = X + Grid = XGrid!

[ctb]

http://a1600.g.akamai.net/7/1600/51/234d85a322f38b/www.apple.com/xserve/images/cluster_031703.gif

These are the special Xserves dedicated to clustering. Is Xgrid just a nickname for that use?

[Rocketman]

Originally posted by ctb
http://a1600.g.akamai.net/7/1600/51/234d85a322f38b/www.apple.com/xserve/images/cluster_031703.gif

These are the special Xserves dedicated to clustering. Is Xgrid just a nickname for that use?

There seem to be functional differences between clusters, grids and farms but getting some to define it seems problematic.

Also there seems to be different OS level methodologies such as MPI drivers, MP hardware, Pooch, and many others as means to either parallelize the task or put alot of simeltaneous computing power to a simultaneous task.

Someone write an outline for me please :)

Rocketman

[lmalave]

Originally posted by ctb
http://a1600.g.akamai.net/7/1600/51/234d85a322f38b/www.apple.com/xserve/images/cluster_031703.gif

These are the special Xserves dedicated to clustering. Is Xgrid just a nickname for that use?

Probably the release of XGrid (whatever it happens to be) will be coordinated with the release of the first G5 XServes.

[ObviousTroll]

Originally posted by Powerbook G5
I wonder what Skynet would have been like if it were ran off of Apple instead of Dell...maybe we'll find out... ;)

[ObviousTroll]

Originally posted by Powerbook G5
I wonder what Skynet would have been like if it were ran off of Apple instead of Dell...maybe we'll find out... ;)

Well, for one thing the Terminators would feature a lot more brushed metal.

[dstorey]

this sounds cool...whatever it is, if only because i've just started a MS in distributed computer, and i wonder if its something that might relate to my course... Shame we have a big deal with IBM and (I think) Sun....and not Apple...theres no an Apple to be seen on my campus, in fact my Java lecturer didnt know that apache etc was available on the Mac (and i guess therefore that they are based on BSD now) or that the laptops don't still come in colours like tangerine :rolleyes: Our lab is full of Sun thin client dumb terminals, powered by the grid (i think)

[greenstork]

Originally posted by AidenShaw
Have you seen the actual performance numbers for VTech, vs. the theoretical numbers that came from looking at the spec sheets and multiplying by 1100?

That 17 GFLOP number for VTech was calculated before the boxes were even delivered.

The "Top500" list, however, requires that you actually plug the computers into an electrical outlet and measure the performance. We don't know what percentage of "17" that number will be....

Considering that the results had to be tested and submitted by October 1, that 17.6 # could be correct, you or I have no idea but there is no reason to think that it is pure speculation given the testing date and the publication date of this article from BBC News.

[ffakr]

Originally posted by greenstork
Considering that the results had to be tested and submitted by October 1, that 17.6 # could be correct, you or I have no idea but there is no reason to think that it is pure speculation given the testing date and the publication date of this article from BBC News.
well, there is reason to believe that these aren't actually performance figures...

the number quoted by the BBC basically works out to the max theoretical double precision FP performance capable on a 970 multiplied out by the number of nodes. Someone on slashdot worked out the figures.

[AidenShaw]

Originally posted by D*I*S_Frontman
What will running all 1100 VT Macs on the latest build of Panther do to the "real world" benchmarks if Panther is supposed to offer such huge speed increases to single G5 systems?

The applications used on a cluster like that probably spend almost all of their time in application CPU-bound code.

Any speed improvements in system functions is unlikely to make much difference in those application loops.

If the applications call improved system libraries such as math libraries, you might see some benefits - but that would be rather unpredictable, it would depend on how much time the app spends in the math or other libraries.

Some people have a misconception that everything will be faster under 10.3, but the reality is that YMMV.

[yamabushi]

What is the fastest way to connect a cluster of 3 to 6 PowerMacs? Would connecting them directly to each other using gigabit ethernet and crossover cables be better than using a switch? Is there a big performance hit if there are more nodes than ethernet cards and some traffic has to hop a node in this configuration? Which is faster in the real world - Gigabit ethernet or Firewire800?

[ClimbingTheLog]

Originally posted by yamabushi
Which is faster in the real world - Gigabit ethernet or Firewire800?

I don't have the toys to tell you real-world, but in theory FW800 offers some nice features for clustering:
1) fast protocol - no IP overhead, guaranteed arrival time - latency can kill clusters
2) daisy-chain-able - no need for expensive switches w/ > 2 machines
3) optical - ethernet media transceivers are more expensive; longer distances and RF immunity key to low retransmit rates
4) futureproof - 1394b goes to 3200Mbps with the same protocol over the same optical cable

Disadvantages
1) routing - how do you route a large FW cluster w/o IP? Who makes a FW router/switch?
2) planning - existing mesh optimization algorithms/tools focused on GigE
3) bus contention - super-expensive Gigabit swiches can provide a full 800Mbps, theoretical between any two nodes.

So, in short, really nice for 'small' clusters with relatively little IO (i.e. not a constant full pipe).

[Jeff Harrell]

Originally posted by yamabushi
What is the fastest way to connect a cluster of 3 to 6 PowerMacs?Use Shake or After Effects. Time required: zero.

Seriously, it depends entirely on what you want to do. If you want to run embarrassingly parallel jobs like network renders, Shake and After Effects have the functionality already, built in. How you do it for other applications depends on the job.

[yamabushi]

I was thinking of building my small cluster for financial apps. I would be writing some scripts and using XML to get them to talk to each other. The problem is that I probably won't be able to recompile them for this specific use. I was hoping for a solution that would not be application specific for this reason. It would be ideal if I could somehow get MP aware code to access all of the nodes without modifying the code. My first cluster is meant as a testbed and thus can be very small. Thus 3-6 nodes, 2 processors each.

It appears that GigE with a switch may be the best way to go because it will scale up well. However, I would like to experiment with 3 nodes connected directly to each other without a switch.

[Jeff Harrell]

Originally posted by yamabushi
I was thinking of building my small cluster for financial apps.Why? What financial application is compute-bound?

[ObviousTroll]

Originally posted by Jeff Harrell
Why? What financial application is compute-bound?

The canonical example is calculating retirement benefits on demand. That was actually a big case study for one of the grid software companies.

BTW - I'm currently researching grids for my company; but they are M$ biased to the extreme.