Xgrid - Parallel and Distributed Computing

Apples and Oranges

Originally posted by alandail
$5.2 million for 17.6 TFlops with the G5

Click to expand...

Have you seen the actual performance numbers for VTech, vs. the theoretical numbers that came from looking at the spec sheets and multiplying by 1100?

That 17 GFLOP number for VTech was calculated before the boxes were even delivered.

The "Top500" list, however, requires that you actually plug the computers into an electrical outlet and measure the performance. We don't know what percentage of "17" that number will be....

Re: Re: XCode technology...

Originally posted by cesar
Shake for Macos X as a app called QMaster that renders across a network, searching for idle macs...

Click to expand...

The same program comes with Compressor, which is bundled both with Final Cut Pro and DVD Studio Pro.

Anybody out there have a Shake 3 install they can refer to quickly? If so, please look at the file /etc/qmaster/QmasterApplications.plist. Is the SingleUser element set to true or false?

Do you think VA Tech and Apple had an agreement about XGrid beforehand? It just seems like too amazing of a coincidence if you ask me. An awesome coincidence. VA Tech will definitely be running XGrid.

Originally posted by alandail
look at what a brand new dell based supercomputer costs

http://www.bizjournals.com/austin/stories/2003/09/29/daily37.html

$5.2 million for 17.6 TFlops with the G5
$38 million for 3 TFlops from Dell

Also, built in 3 months vs. a 5 year project. No wonder Va Tech went with Apple.

Click to expand...

The 3 TFlops represent their initial investment of 300 computers. The article states that they will add 200 more computers (for 2 TFlops) within a year. And the article is not clear on what exactly the 5-year period covers; it does not imply, however, that it took them 5 years to put 300 computers on line. And it doesn't also say what the cluster will look like after they've spent all of the $38 million. It could be a 20 TFlop for all we know.

For sure, the G5 cluster represents dramatic savings (in money and time) over previous clusters. We'll see how the value holds up over the next few years. I wouldn't be surprised to see a 2000-node dual 3 ghz cluster in a year. Can you say 40 TFlops for under $10 mil? The Earth Simulator is the current #1 doing about 40 TFlops at a cost of $350 mil. IBM is shooting for 150 TFlops with their Blue Planet which is slated for mid 2005.

Just for fun

Here:

http://don.cc.vt.edu/updatemeeting/slides/IMG_0647.JPG

one can see that the key person behind the VT Teracluster - Srinidhi Varadarajan wears possibly a Newton MP2x00 on he's waist.

Re: Just for fun

Originally posted by ipiloot
Here:

http://don.cc.vt.edu/updatemeeting/slides/IMG_0647.JPG

one can see that the key person behind the VT Teracluster - Srinidhi Varadarajan wears possibly a Newton MP2x00 on he's waist.

Click to expand...

uhh....its the 21st century and the man still has a *MULLET*!!

best keep him out of the PR shots...

Mullets have been back a while bro...

All the St Martins Fashion students have been sporting mullets for a year nowas have the Paris fashion gurus...probably on their way out AGAIN...

I see your point though...

Re: Just for fun

Originally posted by ipiloot
Here:

http://don.cc.vt.edu/updatemeeting/slides/IMG_0647.JPG

one can see that the key person behind the VT Teracluster - Srinidhi Varadarajan wears possibly a Newton MP2x00 on he's waist.

Click to expand...

And what's definitely a non-Apple computer laptop.

VTech #3 at Rpeak, not #2

Originally posted by Hawthorne
Slashdot has news that the Va. Tech supercomputer (nicknamed "Big Mac") is now the #2 dawg out there. Wow.

Click to expand...

The #2 computer on the list has an Rpeak of over 20 GFlops, more than the VTech cluster. Slashdot made the mistake of comparing the theorical performance of the Mac cluster against the measured performance of the computers on the list.

Of course, the list is sorted by measured maximum performance, not theoretical peak - so we don't know where the VTech will end up if it makes this fall's list....

Originally posted by Sherman
Do you think VA Tech and Apple had an agreement about XGrid beforehand? It just seems like too amazing of a coincidence if you ask me. An awesome coincidence. VA Tech will definitely be running XGrid.

Click to expand...

No, Virginia Tech wrote their own software called Deja Vu. Maybe Apple cut a special deal with Va. Teck so that they could get Va. Tech's code.

What will running all 1100 VT Macs on the latest build of Panther do to the "real world" benchmarks if Panther is supposed to offer such huge speed increases to single G5 systems?

If XGrid turns out to be a fancy way to spread processing threads across several G5 systems (regardless of application), will we soon be entering the "supercomputing cluster for the rest of us" era? Or does every application have to be coded for multiple-system processing distribution?

It would be great to chew through high-def video editing with a rack of even a handful of DP G5 XServes. Time is money for audio engineering and video editing--@ $100/hr starting, how long does it take to justify spending $15K or so on such a system? Answer: less than 4 weeks @ 40hrs/week, or for those who do about 12 large scale projects per year, finishing each project two days ahead of schedule due to productivity increases. Cool.

So if this were the case, would we all someday be buying computers that are merely terminals for accessing a vast processing pool instead of how we do it today with picking one to fit our needs with their own CPU(s)? It would be interesting if 10 years from now we don't even have G4s, G5s, G6s in our computers, but simply fibre channels, wireless networks, and these massive distributed grids where we tap into all of the vast channels of processing power. Instead of saying "I have a dual 2 GHz G5, baby!" we could say "I have access to 6,000 quantum core G8s, baby!" Mmm...teraflops...

Greetings! <p> I have been told by Oracle that next version of their Database software 10g will be available on OS X in Enterprise (vs. current Developer) format. By the way, "g" stands for Grid Computing. Coincidence? Think NOT!

Originally posted by tace
Greetings! <p> I have been told by Oracle that next version of their Database software 10g will be available on OS X in Enterprise (vs. current Developer) format. By the way, "g" stands for Grid Computing. Coincidence? Think NOT!

Click to expand...

And the roman numeral for 10 is... X!

10g = 10 + g = X + Grid = XGrid!

What about these...



These are the special Xserves dedicated to clustering. Is Xgrid just a nickname for that use?

Re: What about these...

Originally posted by ctb

These are the special Xserves dedicated to clustering. Is Xgrid just a nickname for that use?

Click to expand...

There seem to be functional differences between clusters, grids and farms but getting some to define it seems problematic.

Also there seems to be different OS level methodologies such as MPI drivers, MP hardware, Pooch, and many others as means to either parallelize the task or put alot of simeltaneous computing power to a simultaneous task.

Someone write an outline for me please

Rocketman

Re: What about these...

Originally posted by ctb

These are the special Xserves dedicated to clustering. Is Xgrid just a nickname for that use?

Click to expand...

Probably the release of XGrid (whatever it happens to be) will be coordinated with the release of the first G5 XServes.

Skynet...

Originally posted by Powerbook G5
I wonder what Skynet would have been like if it were ran off of Apple instead of Dell...maybe we'll find out...

Click to expand...

Gah. Try that again.

Originally posted by Powerbook G5
I wonder what Skynet would have been like if it were ran off of Apple instead of Dell...maybe we'll find out...

Click to expand...

Well, for one thing the Terminators would feature a lot more brushed metal.

this sounds cool...whatever it is, if only because i've just started a MS in distributed computer, and i wonder if its something that might relate to my course... Shame we have a big deal with IBM and (I think) Sun....and not Apple...theres no an Apple to be seen on my campus, in fact my Java lecturer didnt know that apache etc was available on the Mac (and i guess therefore that they are based on BSD now) or that the laptops don't still come in colours like tangerine Our lab is full of Sun thin client dumb terminals, powered by the grid (i think)

Re: Apples and Oranges

Originally posted by AidenShaw
Have you seen the actual performance numbers for VTech, vs. the theoretical numbers that came from looking at the spec sheets and multiplying by 1100?

That 17 GFLOP number for VTech was calculated before the boxes were even delivered.

The "Top500" list, however, requires that you actually plug the computers into an electrical outlet and measure the performance. We don't know what percentage of "17" that number will be....

Click to expand...

Considering that the results had to be tested and submitted by October 1, that 17.6 # could be correct, you or I have no idea but there is no reason to think that it is pure speculation given the testing date and the publication date of this article from BBC News.

Re: Re: Apples and Oranges

Originally posted by greenstork
Considering that the results had to be tested and submitted by October 1, that 17.6 # could be correct, you or I have no idea but there is no reason to think that it is pure speculation given the testing date and the publication date of this article from BBC News.

Click to expand...

well, there is reason to believe that these aren't actually performance figures...

the number quoted by the BBC basically works out to the max theoretical double precision FP performance capable on a 970 multiplied out by the number of nodes. Someone on slashdot worked out the figures.

Probably not much...

Originally posted by D*I*S_Frontman
What will running all 1100 VT Macs on the latest build of Panther do to the "real world" benchmarks if Panther is supposed to offer such huge speed increases to single G5 systems?

Click to expand...

The applications used on a cluster like that probably spend almost all of their time in application CPU-bound code.

Any speed improvements in system functions is unlikely to make much difference in those application loops.

If the applications call improved system libraries such as math libraries, you might see some benefits - but that would be rather unpredictable, it would depend on how much time the app spends in the math or other libraries.

Some people have a misconception that everything will be faster under 10.3, but the reality is that YMMV.

What is the fastest way to connect a cluster of 3 to 6 PowerMacs? Would connecting them directly to each other using gigabit ethernet and crossover cables be better than using a switch? Is there a big performance hit if there are more nodes than ethernet cards and some traffic has to hop a node in this configuration? Which is faster in the real world - Gigabit ethernet or Firewire800?

Originally posted by yamabushi
Which is faster in the real world - Gigabit ethernet or Firewire800?

Click to expand...

I don't have the toys to tell you real-world, but in theory FW800 offers some nice features for clustering:
1) fast protocol - no IP overhead, guaranteed arrival time - latency can kill clusters
2) daisy-chain-able - no need for expensive switches w/ > 2 machines
3) optical - ethernet media transceivers are more expensive; longer distances and RF immunity key to low retransmit rates
4) futureproof - 1394b goes to 3200Mbps with the same protocol over the same optical cable

Disadvantages
1) routing - how do you route a large FW cluster w/o IP? Who makes a FW router/switch?
2) planning - existing mesh optimization algorithms/tools focused on GigE
3) bus contention - super-expensive Gigabit swiches can provide a full 800Mbps, theoretical between any two nodes.

So, in short, really nice for 'small' clusters with relatively little IO (i.e. not a constant full pipe).