Xserve Clustering

[MacRumors]

Apple's Xserve page (http://www.apple.com/xserve/) reflects their new offering of a "clustering" Xserve configuration:

Designed for the computational clusters and distributed applications, this Xserve configuration delivers high-density processing power ? without the server features you won?t need in a cluster enviroment ? for a price that?s easy to multiply across your deployment

Internetnews.com (http://siliconvalley.internet.com/news/article.php/2114301) details some of the changes (But unlike the original Xserve, which first shipped in July, 2002, the new server has only one hot-swappable 60GB hard drive, no graphics card, and no CD ROM. ):

But unlike the original Xserve, which first shipped in July, 2002, the new server has only one hot-swappable 60GB hard drive, no graphics card, and no CD ROM.

[G4scott]

This is cool. It means that people on budgets, like universities, will be able to afford xServe clusters. Maybe we could pitch in, and buy an xServe cluster for MacRumors folding @ home...

[Stike]

Is it now time for XGrid???

[MrMacMan]

Originally posted by Stike
Is it now time for XGrid???

Um... no.

Yeah how bout $10-$50 each person gives for the folding cluster. :D

Good idea but I don't have soo much money to burn.

I'm glad apple is looking for a new market for these Xserve's. :p

Originally posted by Macrumors
...details some of the changes (But unlike the original Xserve, which first shipped in July, 2002, the new server has only one hot-swappable 60GB hard drive, no graphics card, and no CD ROM. ):

Your URL is messed up for that second link.

[Catfish_Man]

Originally posted by Stike
Is it now time for XGrid???

Probably. The kernel support is there (mach is frickin' awesome for clusters from what I've read), Rendezvous is there, all that I can see that's really needed is GUI configuration tools and some sort of thing that decides whether or not a task is going to take long enough to be worth sending to another machine (some sort of really fancy load balancer, that can deal with drastically different latencies for different processors). I wouldn't expect it immediately, but I would expect it at some point in the relatively near future.

[NavyIntel007]

I think that ideally for the grid you have one full Xserve with many cluster Xserves added on (one would really only need one to have a video card and cd drive).

[Stike]

Originally posted by Catfish_Man
I wouldn't expect it immediately, but I would expect it at some point in the relatively near future.

Whoa, if only I could see the near future! :D
Wait! I have a vision! PPC970-XServes connected via Grid! Whooohooo!

[macman_g4]

Does this mean that OS X.2 has built-in clustering software or are they relying on someone else to make this work?

I remember looking around on the internet awhile ago and found a clustering program called POOCH (http://www.daugerresearch.com/pooch/whatis.html).

[nighthawk]

From the Apple website store: (standard)

$2,799.00

Dual 1.33GHz PowerPC G4
2MB L3 cache per processor
256MB DDR333 SDRAM
60GB ATA/133 ADM
Mac OS X Server (10 client)
Gigabit Ethernet

This makes it the same price as the single processor XServe without all of the extra stuff like a video card, CD-ROM, and secondary Gigibit-Ethernet port. Of course, Education and corporate discounts would bring the price even lower.

What isn't clear is wheither or not this has an AGP or not. The datasheet says that it only has two PCI (64bit) while the same datasheet says that the standard XServe has two PCI (64bit) and one PCI/AGP card. Does this mean that it is a different motherboard, or that they just disabled the slot?

[Rincewind42]

Originally posted by nighthawk
What isn't clear is wheither or not this has an AGP or not. The datasheet says that it only has two PCI (64bit) while the same datasheet says that the standard XServe has two PCI (64bit) and one PCI/AGP card. Does this mean that it is a different motherboard, or that they just disabled the slot?

The XServe has a combo AGP/PCI slot along with the 2 PCI 64/66 slots. Depending on if you get the video card option or the 2nd gigabit ethernet card you will either get an AGP/AGP or AGP/PCI riser card that fits that slot. I remember being able to get the riser card seperately in the past, but I may either be completely wrong, it may no longer be available, or (most likely) I don't know where to look. Since it would be more expensive to cripple the system, I doubt that the cluster version has that slot disabled, at most it may have a different back panel that prevents the slot from being used.

[nero007]

http://a1600.g.akamai.net/7/1600/51/234d85a322f38b/www.apple.com/xserve/images/cluster_031703.gif

[Catfish_Man]

Originally posted by macman_g4
Does this mean that OS X.2 has built-in clustering software or are they relying on someone else to make this work?

I remember looking around on the internet awhile ago and found a clustering program called POOCH (http://www.daugerresearch.com/pooch/whatis.html).

No, 10.2 doesn't have built in clustering software (although the 10.2 kernel could be used as a base for clustering software). Pooch is the only Mac clustering software I've run accross, but I seem to remember that several other programs can cluster ( I think Oracle can for example).

[kansaigaijin]

the second page of the tech sheet says the cluster unit has no pci/agp slot.

"The cluster-optimized Xserve configuration features a single drive bay, one Giga-bit ethernet interfaace, and no optical drive, PCI/AGP slot, or graphics card. See hardware configuration for details.

and clustering software, note the date;
http://www.lindaspaces.com/news/news071901.html

Scientific's Linda® Now Available for Mac OS X
New Haven, CT, July 19, 2001: Scientific Computing Associates (SCIENTIFIC), experts in parallel and distributed computing with more than 21 years of experience, today announced Linda for Mac OS X. SCIENTIFIC's Linda is a unique programming tool which allows developers to parallelize existing applications to run efficiently on a parallel computational cluster. SCIENTIFIC is committed to continue development on current and future versions of Mac OS X.

"Linda-enabled applications running on Mac clusters typically scale almost linearly," said Beverly Thalberg, CEO and President of SCIENTIFIC. "Users can speed their times to solution and also attack problems which have been intractable up to now."

"Power, stability and elegance make Mac OS X the ideal platform for programming and scientific computing," said Ron Okamoto, Apple's vice president of Worldwide Developer Relations. "SCIENTIFIC's Linda software and Mac OS X make the ultimate clustering solution for high-end, clustered computing environments and we are delighted that SCIENTIFIC is bringing this capability to the marketplace."

Linda, introduced in the mid-1980s, was the first commercial product to implement virtual shared memory (VSM) for supercomputers and large workstation clusters. The Linda VSM is content-addressable, not address-based, which makes it easy to build applications, and fully utilize hardware capacity. Cost-effectiveness, speed and ease of use are just some of advantages Linda provides.

Linda for Mac OS X extends the traditional elegance and usability of the Mac platform to clusters or networks of Macs for use as parallel supercomputers. With Linda and Mac OS X, developers now are able to create exciting new parallel applications; they will use clusters in ways previously unforeseen.

Worldwide distribution of Linda for Mac OS X is from Scientific Computing Associates, New Haven, CT www.lindaspaces.com Voice 203-777-7772, Fax 203-776-4074. Pricing starts at $1500.

[ffakr]

Originally posted by Catfish_Man
No, 10.2 doesn't have built in clustering software (although the 10.2 kernel could be used as a base for clustering software). Pooch is the only Mac clustering software I've run accross, but I seem to remember that several other programs can cluster ( I think Oracle can for example).

There isn't clustering software per se, but as far as I understand.. there are some standard shared libraries to facilitate clustering support for apps.
That is, some apps like Blast have the library support already to run a single query with the processing power of many macintoshes.

I expect that Apple will release a more robust clustering solution in software soon. These boxes aren't due to ship in 6 - 8 weeks.. which would be around WWDC. Hmnnn... maybe clustering at WWDC? They do have much more 'enterprise' type tracts this year, though they haven't given details about sessions yet.

[ffakr]

Originally posted by Catfish_Man
Probably. The kernel support is there (mach is frickin' awesome for clusters from what I've read)

One of the things a kernel (microkernel in this case) does is assign threads to processors. In a dual processor machine, the Kernel monitors the load and assigns new threads to the most available processor.
Mach has the really cool ability to extend this over IP networks. It is just as easy for Mach to send a thread to another CPU on another machine as it is for it to send it to the second cpu of the local machine.
Supposedly the NeXT boxes could do this... you could have all the office machines work on your project after hours by configuring them to accept network threads.

.. at least thats how I understand it... I don't write code in the kernel space. :confused:

[Catfish_Man]

Originally posted by ffakr
One of the things a kernel (microkernel in this case) does is assign threads to processors. In a dual processor machine, the Kernel monitors the load and assigns new threads to the most available processor.
Mach has the really cool ability to extend this over IP networks. It is just as easy for Mach to send a thread to another CPU on another machine as it is for it to send it to the second cpu of the local machine.
Supposedly the NeXT boxes could do this... you could have all the office machines work on your project after hours by configuring them to accept network threads.

.. at least thats how I understand it... I don't write code in the kernel space. :confused:

That's roughly the way I understand it too.

Check this out: http://www.barrera.org/machdmmp/machdmmp.htm :)

[kansaigaijin]

and for the less technically inclined, from Apples OSX Server pages, some links,

http://daugerresearch.com/pooch/whatis.html

http://www.gridiron-networks.com/product/overview.asp

http://www.tech-sol.com/

[phrantic]

GridIron actually presented at a recent local apple-sponsored event. Showed some parallelized mpeg4 compression and other nifty tricks.

The trouble with the idea of using the kernel and assigning threads to machines over IP is that there couldn't be any inter-thread data transactions (sending data to each other for tightly integrated parallel problems) and for discovery and authentication on other machines. Kernel space just isn't an appropriate place for this level of interaction ... talk about a way of making the kernel run "slower". Certainly, however, the kernel is optimized to spread threads on SMP machines, but over IP would be backwards due to the overhead of IP based transactions (you think swapping processes and memory in and out takes a long time...)

[conceptdev]

Render Farm

Unlimited number of networked rendering nodes on similar operating systems can be used for free with each Maya Complete software installation

Link (http://www.aliaswavefront.com/en/products/maya/complete/rendering.shtml)

[Rocketman]

Originally posted by NavyIntel007
I think that ideally for the grid you have one full Xserve with many cluster Xserves added on (one would really only need one to have a video card and cd drive).

It seems like a grid of 1-4 boxes would be a good "accessory" for a Powermac or a Powerbook :)

Rocketman

http://www.v-serv.com/-upload/avatar.jpg

[kansaigaijin]

log onto your accessory cluster and see how your maya render is doing,

[etype]

BTW, the "Mac OS X Server Maintenance Program" is $499 rather than $999 for the other configurations...

[kansaigaijin]

that's the plan that includes a spare motherboard right?

[Catfish_Man]

Originally posted by phrantic
GridIron actually presented at a recent local apple-sponsored event. Showed some parallelized mpeg4 compression and other nifty tricks.

The trouble with the idea of using the kernel and assigning threads to machines over IP is that there couldn't be any inter-thread data transactions (sending data to each other for tightly integrated parallel problems) and for discovery and authentication on other machines. Kernel space just isn't an appropriate place for this level of interaction ... talk about a way of making the kernel run "slower". Certainly, however, the kernel is optimized to spread threads on SMP machines, but over IP would be backwards due to the overhead of IP based transactions (you think swapping processes and memory in and out takes a long time...)

There are a lot of tasks (raytracing comes to ming), that don't require much (if any) interthread communication. All you really need is something that tells the kernel whether it's worth it to ship the thread out over the network. For most, it wouldn't be, bu for certain tasks it would be a big win. BTW, this isn't just an idea, Mach already does this, and was designed to do this. It's just not used in OSX yet.

[ffakr]

Originally posted by Catfish_Man
There are a lot of tasks (raytracing comes to ming), that don't require much (if any) interthread communication. All you really need is something that tells the kernel whether it's worth it to ship the thread out over the network. For most, it wouldn't be, bu for certain tasks it would be a big win. BTW, this isn't just an idea, Mach already does this, and was designed to do this. It's just not used in OSX yet.

yea, video rendering is just too perfect for distributed processing. It's fairly trivial to assign sections of a an image to one system, or objects in a frame.. or even alternate frames to different systems.
You don't have to get any feedback from the other machines until they are done.. you just need one box to manage everyting and reassemble the finished data.

[Dj Kioto]

wouldn't this mean that it could offer cluster across G3s, iMacs, and any other computer if they are all using the same version of OS X, assuming said version actually supports clustering capabilities

[Dj Kioto]

wouldn't this mean that it could offer cluster across G3s, iMacs, and any other computer if they are all using the same version of OS X, assuming said version actually supports clustering capabilities

[PyroTurtle]

i'll take 42 xserves clusterd together then...
yay for folding then!
have fun dreaming everyone!

[Quixcube]

Anyone up to snuff on the idea behind Sony's new PS3 network computer grid design? From what I understand, the idea is that as PS3s come online, they offer up their available processing power and combine to create a grid computer capable of doing things that a lone PS3 couldn't manage. Machines share processor cycles etc. (Is this accurate at all? I am not a PS3 rumor person.) So now on to Apple :)

I wonder if Apple is considering something like this for their machines. Xserves first, but maybe all of their machines further down the road. At any given time across the campus where I work, many Macs are sitting idle or mostly idle while others are chugging away at some pretty impressive calculations. If some friendly technology like Rendezvous could help the Macs that are swamped find the macs that are idle, and if the software existed in the OS to help share their processing power, interesting things could happen.

Of course, the machines could only be as interactive as their connections to each other are fast, but the possibility is exciting. In an ideal environment (switched office, campus, server farm), you could increase the power of all of your existing Macs by adding new Macs to the network. That's investment protection for you.

On a more immediate note, I set up one of the new dual 1.33Ghz Xserve's this week and was stumped by a third "ethernet" interface that shows up in the Network Port Configuration settings. This port wasn't available on the first generation Xserves. I asked one of Apple's support people about it (while he was helping me track down some evil Netboot problems) and he said that the third ethernet interface actually refers to one of the new firewire 800 ports on the back of the machine. The idea being that a cluster can have firewire ethernet connections, leaving the standard ethernet interfaces free for Netbooting, web hosting, etc. The new firewire ethernet port can be configured with all of the settings that the other conventional ports have, although I really don't have a way to try it out. I would be willing to bet that both firewire interfaces could be used just as easily in the future for a *very* well connected cluster.

All of this computer combining makes me want to dig out my old Voltron :)

[Catfish_Man]

Originally posted by Quixcube
Anyone up to snuff on the idea behind Sony's new PS3 network computer grid design? From what I understand, the idea is that as PS3s come online, they offer up their available processing power and combine to create a grid computer capable of doing things that a lone PS3 couldn't manage. Machines share processor cycles etc. (Is this accurate at all? I am not a PS3 rumor person.) So now on to Apple :)

I wonder if Apple is considering something like this for their machines. Xserves first, but maybe all of their machines further down the road. At any given time across the campus where I work, many Macs are sitting idle or mostly idle while others are chugging away at some pretty impressive calculations. If some friendly technology like Rendezvous could help the Macs that are swamped find the macs that are idle, and if the software existed in the OS to help share their processing power, interesting things could happen.

Of course, the machines could only be as interactive as their connections to each other are fast, but the possibility is exciting. In an ideal environment (switched office, campus, server farm), you could increase the power of all of your existing Macs by adding new Macs to the network. That's investment protection for you.

On a more immediate note, I set up one of the new dual 1.33Ghz Xserve's this week and was stumped by a third "ethernet" interface that shows up in the Network Port Configuration settings. This port wasn't available on the first generation Xserves. I asked one of Apple's support people about it (while he was helping me track down some evil Netboot problems) and he said that the third ethernet interface actually refers to one of the new firewire 800 ports on the back of the machine. The idea being that a cluster can have firewire ethernet connections, leaving the standard ethernet interfaces free for Netbooting, web hosting, etc. The new firewire ethernet port can be configured with all of the settings that the other conventional ports have, although I really don't have a way to try it out. I would be willing to bet that both firewire interfaces could be used just as easily in the future for a *very* well connected cluster.

All of this computer combining makes me want to dig out my old Voltron :)

It SEEMS like it ought to be relatively easy to set up some sort of unintrusive autoclustering that just borrows free CPU time from other computers on the network. Rendezvous would find the computers, and mach would share threads. The trick would be deciding which threads to share based on network latency, without using too much CPU time for the task doing the deciding. If it worked, then each new Mac plugged into the network (if it had the "share processor(s) over network" checkbox checked) would speed up all the other machines slightly. It would be *great* for schools; think about a 3D graphics class where absent student's computers were "borrowed" for renders, and kids could come in before or after class to use the whole cluster. <drools> :)

btw, WWDC is going to have a session on clustering.

[hotFusion]

I have recently been charged with upgrading all of the systems in our office (which currently consists of Mac SEs/Performa 4xx and Laserwriter IIs which are passed being on their last legs and are actually hovering in mid-air) and this entire discussion intrigues me deeply. The idea that all of our machines, which are going to networked with GB Ethernet (it's there, so why not), could be harnessed so seamlessly is very appealing in deed. Most of what we do is publishing, and most of the machines will be used to simply write articles, but the idea that the couple of machines that will be used to render images and video could use the other machines when they are not in use (which is quite often considering their schedules) makes me very interested in the future of this technology (;) hmm).