XGrid- What's all the buzz about?

[jbouklas]

Remember when XGrid was announced a year and a half ago? There was tons of buzz about linking up all the Macs in your house to render something in Final Cut Pro or working with Photoshop at lightning speeds. Then, everybody got real quiet about it.

Well, Tiger shipped, and according to Jobs, they shipped over 2 million copies of the damned thing, each one including an XGrid client. Now, with the mass distribution of this "spectacular piece of technology," what good is it? I've Googled XGrid plugins and whatnot every couple of weeks for the past few months, and there's nothing new. No plugins for any apps, except for a few specific number-crunching research programs.

Everybody was talking about this new revolution in computing and now that the technology is widely distributed, can anybody find a single use for it? I've been dying to set up a cluster of Macs in my house to convert large MPEG files over to Divx, but haven't found anybody even working on anything close to this.

So, if anybody has any info on this, please post here.

 

[killuminati]

I'm sorry, I have nothing to add to this thread, I just want to subscribe to it to find out what answer you get. I am also really interested in this.

 

[live4ever]

Apple took this out of Tiger client, Tiger server is needed to set up the grid I believe.

Distributed computing is built into DVD SP 4 though for encoding video.

 

[Mechcozmo]

My friend and I tried the demo. Couldn't figure out anything to supercompute, though.

 

[Soulstorm]

Here is what I found in looking my Tiger's help server:

Xgrid provides the easiest way to run computer-intensive applications. Xgrid allows computers on a network to work together in a grid to process a job. Administrators of Mac OS X Server can group locally networked computers, or "nodes," into grids (also called clusters). Every computer using Mac OS X v10.3 or later can participate in a grid, using the Xgrid agent that is part of the system software.

Xgrid helps scientists and others working in compute-intensive environments fully utilize all computing resources, including desktops and servers. By creating a grid, an administrator can take advantage of unused computing capacity to run batch and workload processing. An example of a project that works the way Xgrid does is the University of California at Berkeley's Seti@Homeproject. Seti allows computer users around the world to donate processing time for analyzing radio telescope data on behalf of the project team’s search for extraterrestrial intelligence.

The Xgrid software uses three components in a grid. The agent computers perform the tasks in a job; the controller sets up a grid and controls it; and the client submits jobs to the grid.

The Xgrid agent uses Bonjour networking technology to discover and connect to a grid. You turn the Xgrid agent on or off in the Services pane of Sharing preferences.

Mac OS X lets you set up your computer as an agent and accept jobs from controllers on the network you are connected to......

and somewhere in the end it says:

To use your computer as an Xgrid controller you need to install Mac OS X Server. To submit jobs to a grid, use the Xgrid command-line tool. For more information visit the Xgrid website.

 

[Darwin]

I'm also interested in seeing what happens to Xgrid, a lot of people have more then one machine in their houses now so it would be nice to see that resource used, especially for things like media encoding

Supercomputing for the rest of us

 

[jbouklas]

Cool

Wow. Looks like a lot people are in the same boat. Lots of questions so far, but no answers.

 

[yellow]

xgrid still exists in the Tiger client:

[null:~] yellow% sw_vers
ProductName:    Mac OS X
ProductVersion: 10.4.1
BuildVersion:   8B15
[null:~] yellow% locate xgrid
/Developer/Examples/Xgrid/GridSample/MPI/xgridmpiboot
/Library/Preferences/com.apple.xgrid.agent.plist
/Library/Preferences/com.apple.xgrid.controller.plist
/private/etc/xgrid
/private/etc/xgrid/agent
/private/etc/xgrid/agent/com.apple.xgrid.agent.plist.default
/private/etc/xgrid/controller
/private/etc/xgrid/controller/com.apple.xgrid.controller.plist.default
/private/var/xgrid
/private/var/xgrid/agent
/private/var/xgrid/agent/cookies
/private/var/xgrid/controller
/private/var/xgrid/controller/blobs
/System/Library/LaunchDaemons/com.apple.xgridagentd.plist
/System/Library/LaunchDaemons/com.apple.xgridcontrollerd.plist
/usr/bin/xgrid
/usr/libexec/xgrid
/usr/libexec/xgrid/DatasetTool
/usr/libexec/xgrid/execuid
/usr/libexec/xgrid/IdleTool
/usr/libexec/xgrid/xgridagentd
/usr/libexec/xgrid/xgridagenthelper
/usr/libexec/xgrid/xgridcontrollerd
/usr/sbin/xgridctl
/usr/share/man/man1/xgrid.1
/usr/share/man/man8/xgridctl.8

I think most of it is, if you don't have anything super-processor-intensive (something that takes multiple days to compile/render/whatever), you wouldn't have any need for xgrid. Install the Dev tools, look at the examples, read the man pages..

 

[jbouklas]

So, now what?

Okay, it looks like the support is in Tiger, there are tutorials or whatnot in XCode, and there's a large user base. Now what? We are mostly laypeople here who want to download a binary of an application and run it with an XGrid plug-in as part of it. Is it really possible that nobody has been developing this at all?

Take Divx movie encoding. A big bottleneck is CPU usage. Most new Macs have Gigabit networking (10/100/1000), allowing for very high file transfer speeds, meaning that even file-intensive tasks would benefit from XGrid. As an example, you have an MPEG stream of roughly 1.5 gigabytes. On my computer, it takes over an hour and a half to convert to good quality XVid. If I could split this file into 300 MB chunks and have it distrubted across my 5 workstations in my house, I could conceivably cut this time dramatically. The only limitation would be file transfer speed, and I don't perceive this as a problem. So, while you pointed out that this can be done, etc, is there any work actually being done on it?

 

[yellow]

So, while you pointed out that this can be done, etc, is there any work actually being done on it?

Who knows. Sounds like you should get started soon though, it's a good idea.

Maybe you could also subscribe to the Apple Xgrid users mailing list, as well.

 

[jbouklas]

...

As software developing isn't my forte, I doubt that "getting started" would help much. As consumers, we say what we want, and developers make it. Then, to keep the whole process nice and good, we pay them for the product.

It looks like there isn't much of a community pull to get this technology in applications. If there was, I suppose we'd have software titles that actually used the full technology of the OS. "Supercomputing for the rest of us" was used to help sell us Tiger and now Apple and other developers have to start utilizing this framework if they want to follow through on their marketing. Honesty is always a good policy when it comes to delivering a good product.

 

[yellow]

It looks like there isn't much of a community pull to get this technology in applications. If there was, I suppose we'd have software titles that actually used the full technology of the OS.

I'm not so sure about that. If you Google, you will find some XGrid plug-ins. I think what you're missing is that most people don't have multiple Macs, and have no need for XGrid. Those that need it recognize this and either find the plug-ins they need, create them if they don't exist, or contract someone to do it for them.

Hell, most Mac users don't know anything about the CLI, and don't want to.

 

[redeye be]

I'm sorry, I have nothing to add to this thread, I just want to subscribe to it to find out what answer you get. I am also really interested in this.

ah, I know the feeling.
However, there is a solution!

Click 'thread tools' in the blue ruler on top of the thread. You'll conveniently find a 'Subscribe to this thread' link.

Cheers

 

[sigamy]

Take Divx movie encoding. A big bottleneck is CPU usage. Most new Macs have Gigabit networking (10/100/1000), allowing for very high file transfer speeds, meaning that even file-intensive tasks would benefit from XGrid. As an example, you have an MPEG stream of roughly 1.5 gigabytes. On my computer, it takes over an hour and a half to convert to good quality XVid. If I could split this file into 300 MB chunks and have it distrubted across my 5 workstations in my house, I could conceivably cut this time dramatically. The only limitation would be file transfer speed, and I don't perceive this as a problem. So, while you pointed out that this can be done, etc, is there any work actually being done on it?

I have a very similar need. I've been wanting to share iDVD encoding across multiple Macs ever since I've heard about Xgrid. Now DVD Studio Pro has this feature (distributed encoding). I'm waiting for some brilliant developer to release Xgrid plugins for iMovie and iDVD. I'd love to cut my DVD encoding time down.

And it looks like we don't even need OS X Server, check out:
http://www.edbaskerville.com/software/xgridlite/

 

[jbouklas]

I disagree with the point that most people only have one computer at home. There is a very big user base that has multiple computers in the house that are networked. Otherwise wireless routers and the like wouldn't be so popular...

Besides that, I have scoured Google looking for XGrid plug-ins and have found little more than proof of concept. Where is everything? Is there at least one program, other than DVD SP, that has XGrid support? Just one?

 

[killuminati]

ah, I know the feeling.
However, there is a solution!

Click 'thread tools' in the blue ruler on top of the thread. You'll conveniently find a 'Subscribe to this thread' link.

Cheers

Ahh, I never knew that.

Thank you

 

[homerjward]

sounds pretty cool, except i would have no use for it, or any idea of how to do so
what's next though, iGrid--supercomputing for the rest of us?

 

[yellow]

I see you want to stick to your guns. Well, good luck with it.