Coming Soon: Mac Server Clusters

bidge

macrumors 6502
Jul 16, 2002
286
0
New Zealand
i was just talking about this with a friend the other day, he said that OS X is the only UNIX based OS that doesn't do this? Is this bull or is it true?
 

sparkleytone

macrumors 68020
Oct 28, 2001
2,307
0
Greensboro, NC
now this is something i hope is true. apple could really pull a fast one on everyone if they can come up with a classic apple solution to clustering. the ease of use and capabilities in an implementation of this could mean wonders.
 

bidge

macrumors 6502
Jul 16, 2002
286
0
New Zealand
Or if they used rendesvouz so that they automatically discovered each other you could go into an Apple store and have all of the computers clustered.

Awesome rendering capabilities. You could be working on your powerbook while your dualie's rendering.

Cool
 

oldMac

macrumors 6502a
Oct 25, 2001
522
1
You can cluster just about anything...

Software exists to cluster almost any type of Unix, NT or even Mac OS 9 servers.

The hard part is:

1) Making sure that the software you intend to cluster runs on your hardware.
- It's just like any other software, it has to be compiled specifically for the hardware you intend to run it on.
- It's much easier to port Unix software to the Mac now that we've got OS X, so there is a lot more clustering potential for the Mac OS now.

2) Setting up clustering can be a pain
- Generally this process isn't pretty in existing solutions
- Apple could utilize Rendevous to make this a lot easier
- Even just throwing a nice GUI on the setup process would be a huge help

It would be interesting to see if Apple provides a way to automatically distribute the software that you intend to cluster, as well. Imagine an interface that looks like this...

1) You open the "Clustering" control panel
2) Rendevous automatically discovers the local machines available for clustering
3) You can select/deselect the machines available from the list that's displayed
4) You drag your application into the control panel
5) Click "Start"


Since Apple built the XServe for clustering, it really makes sense that they would be getting ready to bundle nice software for it. This is completely supported by including Rendevous in OS 10.2
 

nuckinfutz

macrumors 603
Jul 3, 2002
5,504
314
Middle Earth
Originally posted by bidge
i was just talking about this with a friend the other day, he said that OS X is the only UNIX based OS that doesn't do this? Is this bull or is it true?
Ahhh I don't think he's thinking this through. This is about Clustering at the "Kernel" level. I doubt that this is a standard item of all Unices.
 

MisterMe

macrumors G4
Jul 17, 2002
10,650
28
USA
Originally posted by bidge
i was just talking about this with a friend the other day, he said that OS X is the only UNIX based OS that doesn't do this? Is this bull or is it true?
Your friend is talking through his hat. UNIX is clustered using a tool called MPI, available free of charge from the University of Michigan, IIRC. If all versions of UNIX included clusering as a standard feature, then MPI would never have been developed. BTW, there is a MacOS 9 version of MPI called MacMPI.
 

Chisholm

macrumors regular
May 31, 2002
241
11
Tuscaloosa, Alabama
It seems to me the xServe was released a little prematurely. The very nature of its design (case wise) seems like clustering would be almost an assumed component of the OS.

"Harness all of the power (almost, well kinda' if you look at it from this angle under this lighting, etc.) of Unix"
 

nuckinfutz

macrumors 603
Jul 3, 2002
5,504
314
Middle Earth
Originally posted by Chisholm
It seems to me the xServe was released a little prematurely. The very nature of its design (case wise) seems like clustering would be almost an assumed component of the OS.

"Harness all of the power (almost, well kinda' if you look at it from this angle under this lighting, etc.) of Unix"
Not really. There's Firewire although I doubt it'd be used for clustering. Xserves have PCI and that would probably be your link to PCI Fibre Channel based connections.
 

arn

macrumors god
Staff member
Apr 9, 2001
14,503
1,785
Originally posted by Chisholm
It seems to me the xServe was released a little prematurely. The very nature of its design (case wise) seems like clustering would be almost an assumed component of the OS.
It can cluster... there is software for it. It's simply not built into the kernel.

arn
 

shadowfax0

macrumors 6502
May 2, 2002
408
0
Everyone go look up 'Project Appleseed' in Google, RIGHT NOW. I made a cluster between my G4 tower and a TiBook to render a Mathematica problem...and it still took a week! Anyhow, I suggest everyone go look fo rit, becuase just a few powermacs can equal the power of a Cray T3E.
 

shadowfax0

macrumors 6502
May 2, 2002
408
0
Also, this is a bit of a side note, but I think it applies. All you MacRumors folks who live in England, go to a magazine shop, and get the mag 3D World, they have an article about making a 'render farm' by utilizing Maya, Lightwave, or Cinema 4D, very helpful tutorial, and I know alot you folks are looking for it!
 

P-Worm

macrumors 68020
Jul 16, 2002
2,045
0
Salt Lake City, UT
Originally posted by shadowfax0
Everyone go look up 'Project Appleseed' in Google, RIGHT NOW. I made a cluster between my G4 tower and a TiBook to render a Mathematica problem...and it still took a week! Anyhow, I suggest everyone go look fo rit, becuase just a few powermacs can equal the power of a Cray T3E.
Does this work for OS X yet? I'm excited about the technology, but I want to make myself less OS 9 dependent, not more.

P-Worm
 

shadowfax0

macrumors 6502
May 2, 2002
408
0
AppleSeed is Mac OS X all the way, it has a Mac OS 9 version, so if you have nodes running 9, they can help too :)
 

IJ Reilly

macrumors P6
Jul 16, 2002
17,892
1,466
Palookaville
Originally posted by shadowfax0
AppleSeed is Mac OS X all the way, it has a Mac OS 9 version, so if you have nodes running 9, they can help too :)
Then you really ought to update the Project Appleseed FAQ -- it says no support for OSX yet, and I can't find OSX mentioned anywhere else on the site.

One other gotcha worth advertising: an application needs to be written (or re-written) to take advantage of parallel computing.
 

MacHack

macrumors newbie
Mar 22, 2002
21
0
Chicagoland
Controller needed?

What exactly is in that controller thingy.
It seems to be directly connecting: 1) Firewire, 2) RAM, 3) Processors. Hmmm...
Gee, if only that could help to facilitate linking something that could take advantage of a really fast link to the processor with a 2GB DDR buffer. Hmmm...
I wonder why it showed up in the xServe?
Now that it is in the desktops, I wonder if they, too, could take advantage?
Boy, with faster Firewire and much faster bus speeds coming.... okay, that's enough.
 

Telomar

macrumors 6502
Aug 31, 2002
260
36
This has been in Apple's sights for a very long time now. Back in January/February it was being aimed at the next major release after the current one. Whether or not they are still aiming it for then I can't say.
 

Catfish_Man

macrumors 68030
Sep 13, 2001
2,579
1
Portland, OR
Screw configuration...

...this type of processing (massively parallel, especially clusters) is exactly what Mach was designed for (according to "Building Cocoa Applications" by the O'Reilly group). The kernel level support is all there (you can schedule tasks against "processor groups", which, iirc, can be on a networked machine), Rendezvous is there (for auto-discovery of clusterable machines), all we need is two things: A little check box in the sharing preference, and a thing that checks each task being processed to see if it's worth sending to another machine (really small tasks would probably take MORE time if you sent them to another machine because the network latency would be higher than the processing time). You could even include it in the home OSX. Advertise it as each computer on a network taking advantage of the unused resources of the others (the server version would probably be optimized differently as you wouldn't have to worry about stealing CPU time from programs on the other machine). I really hope they're thinking along these lines... mmmm...home autoclustering networks....

<crosses fingers>

disclaimer: the info about Mach is not guaranteed to be at all accurate, it's a moderately vague memory
 

shadowfax0

macrumors 6502
May 2, 2002
408
0
AppleSeed is OS X supported; I've used it in OS X natively. The program you're looking for is Pooch. You can even get it on versiontracker.com I believe. Hope that helps :) Also, a little disclaimer: This is no 'magic program' that no matter what application you drop on it it's going to spread out over multiple computers, it's not designed like that. The program itself must be WRITTEN to be clustered; with my Mathematica problem, I had to write some part of it in C, but Mathematica is meant for things like this, so that helped out. But things like SETI won't work unless someone wants to roll up their sleeves and take a stab at it (which wouldn't be a bad idea!) Anyhow, good luck, program away! :)
 

sparkleytone

macrumors 68020
Oct 28, 2001
2,307
0
Greensboro, NC
i would assume this would only work on GiG-E configurations of Apple Hardware, with Gig-E being required to functoin. You really wouldnt want any kind of latency issues going on here.
 

MacHack

macrumors newbie
Mar 22, 2002
21
0
Chicagoland
Wouldn't Firewire work better that Gigabit Ethernet for a cluster in the same rack? I would think so. Maybe that could be an option in the setup? Firewire or Ethernet.
What does a fully loaded (42u) rack go for anyway? I need to start budgeting.
 

Telomar

macrumors 6502
Aug 31, 2002
260
36
Gigabit ethernet so no not really.

A full 42U rack would set you back probably around $300k. Assuming all you wanted was standard Xserves.