Portions of Apple's Grand Central Dispatch Go Open Source

[MacRumors]

http://www.macrumors.com/images/macrumorsthreadlogo.gif (http://www.macrumors.com/2009/09/11/portions-of-apples-grand-central-dispatch-go-open-source/)

Daring Fireball's John Gruber noted (http://daringfireball.net/linked/2009/09/10/libdispatch) yesterday that Apple has released libdispatch (http://libdispatch.macosforge.org/), the source code for the user space implementation of Grand Central Dispatch (http://www.apple.com/macosx/technology/#grandcentral), taking a significant portion of the company's technology for more efficiently utilizing multicore processors open source. While some may consider the move a surprise due to the technology's key role in the core of Mac OS X Snow Leopard, MacResearch points out (http://www.macresearch.org/grand-central-now-open-all) several reasons why the move may not be all that risky for Apple.So why did they do it? Only Apple knows for sure, but there are compelling arguments for open sourcing Grand Central Dispatch, even for a commercial enterprise. First, Apple will of course reap the rewards of any development that takes place, just as they have with WebKit. Second, it is unlikely that Grand Central would be used by any direct competitor to Apple, like Microsoft. Grand Central is more likely to be added to other UNIX and Linux systems, none of which really pose a threat to Apple's consumer-based business.Opening up Grand Central Dispatch for broader distribution into the UNIX community could ultimately serve to spark new innovations using the technology that could make their way back to the Mac platform. In addition, wider distribution of the technology could drive the adoption of other technologies such as blocks, the non-standard extension to the C programming language upon which Grand Central Dispatch is based.

Article Link: Portions of Apple's Grand Central Dispatch Go Open Source (http://www.macrumors.com/2009/09/11/portions-of-apples-grand-central-dispatch-go-open-source/)

[thegoldenmackid]

Yay... does that mean we get Blu-Ray support?

[Peace]

Yay... does that mean we get Blu-Ray support?


What it means is Apple needs some help in development.;)

[mattwolfmatt]

As an avid mac user and an absolute programming idiot, all I have to say is "HUH?" :eek:

This is all over my head, but it sounds good (I think.)

[nagromme]

It will also increase awareness/mindshare, which may spur developers to do more with this great tech even in its existing form.

[amac4me]

In my view it is a play against Microsoft Windows, plain and simple.

[Corrosive vinyl]

Grand central from what I have read basically is a way to make more use of multiple core CPUs. This is great because apple has made something new, and to make it more useful they have open sourced it so that regular programmers can take it and use it to make more and better performing mac apps.

[Stetrain]

Grand central from what I have read basically is a way to make more use of multiple core CPUs. This is great because apple has made something new, and to make it more useful they have open sourced it so that regular programmers can take it and use it to make more and better performing mac apps.


Programmers making Mac apps don't need the source to use GCD, its a framework that's available to anyone developing on the platform for 10.6+

I think the benefit will be in continued development in the unix/linux community as a whole as computer hardware changes in the future.

[shawnce]

Grand central from what I have read basically is a way to make more use of multiple core CPUs. This is great because apple has made something new, and to make it more useful they have open sourced it so that regular programmers can take it and use it to make more and better performing mac apps.

GCD is part of Snow Leopard. Mac developers already have access to its facilities, they don't need access to the source code.

This is simply as Apple states to make it available to expand it use outside of just Mac OS X to benefit developers and as a result Apple. Also to see what extensions, enhancements people can think of.

[Chimpy]

...and this is why I love Apple. :)

[NT1440]

...and this is why I love Apple. :)

They do seem to champion open source in many regards.

[corinhorn]

They do seem to champion open source in many regards.

http://www.youtube.com/watch?v=cMPQuv34vxQ

[Shiner]

They do seem to champion open source in many regards.

Really?? Apple champions open source?

[NT1440]

Really?? Apple champions open source?

Ever hear of webkit?

[notjustjay]

Yeah, but there's still no camera in the iPod Touch!






... just kidding :rolleyes:

[Doctor Q]

For those interested in reading about "blocks":
Introducing Blocks and Grand Central Dispatch (http://developer.apple.com/mac/articles/cocoa/introblocksgcd.html)

[ltcol266845]

Out of curiosity, are there any applications taking advantage of OpenCL and GCD? I know its early, just wondering. What about Apple's new creative applications like Final Cut Pro, Motion, Compressor and the gang? Seems those would have the most to gain with OpenCL especially and GCD to a lesser extent (I assume they are already pretty well multi-threaded).

[Spades]

This is the Apple that I love. The one that provides lots of nerd goodies for us to play with. I wish this Apple showed up more often than the control freak Apple.

[VoR]

They do seem to champion open source in many regards.

I think a fair few oss advocates would disagree :P Apple's history is hardly great, with like two exceptions...

They're just using it to their own advantage as they have done many times in the past - any commits from apple to any oss is a 'positive' in some respects, little more.

[SpinThis!]

For the average Mac user this doesn't mean much.

To make this ultimately useful, other OSes would need to bake this into the kernel so we'll see if and when that happens. It took years for WebKit to mature and end up being used by others besides Apple; we'll see how this project fares.

The next step would be for people to actually start using it and that's where it'll get interesting. Say, if projects like x264 adopt this, the trickle down effect would happen and apps like HandBrake would be able to take advantage.

[larrylaffer]

For the average Mac user this doesn't mean much.

To make this ultimately useful, other OSes would need to bake this into the kernel so we'll see if and when that happens. It took years for WebKit to mature and end up being used by others besides Apple; we'll see how this project fares.

The next step would be for people to actually start using it and that's where it'll get interesting. Say, if projects like x264 adopt this, the trickle down effect would happen and apps like HandBrake would be able to take advantage.

I feel WebKit took "years" to mature largely because it took a while before major players decided it was a project worth using for their own needs. Sure it's nice to have the KDE and Symbian folks hacking away at it, but once Google started contributing, the growth accelerated substantially. In the last 6 months of running WebKit nightlies I've seen huge performance gains everywhere. It's very exciting stuff!

I think technologies like GCD are going to play a big role in netbooks and mobile devices since it's intended to allow an application to make use of every available resource efficiently. With Google developing an OS I wouldn't be surprised to see them considering using GCD for themselves, and then contributing great improvements to it (though it may be too late in their dev cycle to make such a big change).

[bigwig]

What does this do for you that pthreads don't?

[Spades]

What does this do for you that pthreads don't?

I saw a good analogy for GrandCentral earlier today. pthreads is to GrandCentral as assembly is to C. It's just far easier to do parallel programming with GrandCentral than it is with pthreads. Except GrandCentral is system wide, so it can tune threads even better than the programmer could.

However, one of the features required for GrandCentral, blocks, is a major new feature for C and C-derived languages. Blocks are closures for C, and closures are a powerful little tool. Apparently, Apple's block extension has been open source and in LLVM for awhile.

[shakenmartini]

This is awesome news. If this filters into Linux, you can bet the Linux and OSX will be even stronger contenders for high performance computing.

Already OSX is becoming a popular platform for scientific computing because it is easy to port Linux programs to OSX.

[mserrano303]

What does this do for you that pthreads don't?

Each application (and therefore the application programmer) has to maintain his own threads with pthreads.

With GCD, the operating system dynamically determines the optimal way to thread all of the currently running applications for you.

[wizard]

I think a fair few oss advocates would disagree :P Apple's history is hardly great, with like two exceptions...

They're just using it to their own advantage as they have done many times in the past - any commits from apple to any oss is a 'positive' in some respects, little more.

This is sad misinformation and unfortunately keeps alive a myth that undermines Apples efforts in this regard. Off the top of my head these are some to the technologies Apple has supported over the years:

gcc, especially the PPC improvements
llvm
clang
cups
webkit
X
ruby


This is just a quick list off the top of my head and thus I urge anybody with more accurate information to add to the list. I'm pretty sure I'm missing stuff. macosforge has a few projects listed there.

Certainly some stuff they do in a way that the open source community objects to. GCD dispatch is an example of a complete project coming to term and no open source involvement. While the open sourcers would prefer to be involved from the get go, it is also obvious that Apple doesn't have that luxury.

Maybe the open source community is upset that Apple doesn't sponsor every project. I'm not sure what the issue is but personally that would be the last thing I'd want to see in the open source arena, we just don't need one company dominating here. There is also the problem of some in the open source community demanding things be done their way or it is not acceptable, that is way to narrow minded and frankly I'm glad we have a plurality of open source licenses to choose from.

In any event I think Apple needs more balanced reporting on this issue.


Dave

[djmcave]

Pthreads cant dispatch to GPU, unless you build the threads to support GPUs and in that case you need to code for Nvidia and ATI separately.
This works out everything that has to do with threads and will even tap resources you had to custom code for before.

[larrylaffer]

This is sad misinformation and unfortunately keeps alive a myth that undermines Apples efforts in this regard. Off the top of my head these are some to the technologies Apple has supported over the years:

gcc, especially the PPC improvements
llvm
clang
cups
webkit
X
ruby


Dave

I completely agree. CUPS is an excellent example of how Apple helped mature a very incomplete and unstable product into something nice and usable.

[rei-t-ex]

This is great news. Much as what happened with WebKit (and in contrast with Opera's still technically brilliant Presto), this strongly increases the likelihood of GCD becoming the standard for multicore implementations, strengthening Apple's position as a result. After a somewhat disappointing Music event, this is an excellent development.

[shamino]

Really?? Apple champions open source?
Ever hear of webkit?
I think a fair few oss advocates would disagree :P
Like all corporations, Apple does what's in the best interest of Apple.

For many components of their products (like their decision to use gcc, or release WebKit), they have decided that it is in their best interest to use open source software. For other components, they've decided that it isn't.

With the exception of a few idealistic/altrustic groups (like the GNU people), this is the way everybody operates. Personally, I'm glad that Apple releases as much source code as they do. Companies like Microsoft and Sun/Oracle release much much less, or they release it with really annoying license terms.

What does this do for you that pthreads don't
Based on the descriptions I've read, it appears that GCD's runtime will use pthreads on operating systems where pthreads is the native threading API. (Although I suppose it might actually call the lower-level APIs that pthreads uses on platforms like Linux, where pthreads runs as a part of the C library.)

Of course, if GCD is ported to a non-pthreads platform, then it would obviously use that platform's native thread APIs.

[Jimmetry]

This is *excellent* news. This is the best solution to parallelism out there by far, and I'm looking forward to using it in the near future.

Going open source is good for Apple for many reasons, but I think the most compelling is this:
Big developers for the science and game industries don't have a lot of reason to create Mac-compatible software due to the small market share. If GCD is adopted in full by the open source community (I think it will, to be honest), it means that the actual market share is now, essentially, everyone except Windows. With more cores being added to computers, and no easy way to utilise them properly in Windows, it may mean that more projects will become focused on having Linux/OS X as the primary platform rather than the afterthought.

While not directly relevant to the open source nature of GCD, I think it's also worth noting that this technology makes OS X an ideal development platform. It would be much easier to develop a multithreaded game on a Mac, work through all the bugs, and then port it across to Windows than to start from scratch from a messy environment. Parallelism is a huge cost to debugging, and the ability to have that stage isolated (squash all other bugs in the software first, then implement manual threading during the port) is quite a compelling advantage.

[bobrik]

drive the adoption of other technologies such as blocks, the non-standard extension to the C programming language

That is the last thing I want, non-standard language extension being adopted.

[gnasher729]

That is the last thing I want, non-standard language extension being adopted.

Non-standard + adoption => standard.

[Jimmetry]

That is the last thing I want, non-standard language extension being adopted.

Uh... pretty sure adoption of non-standards is how a lot of great things become.. you know.. standard.

'Blocks' is something that C desperately needs in order to move into the parallel world... one of few, though.

[ldmbouge]

That is the last thing I want, non-standard language extension being adopted.

GCD is useless without the block extension. I, for one, hope that Linux adopts it (both GCD and the blocks extensions). Think of it this way, blocks are just closures (in the functional sense). So it is extremely valuable to have.

What GCD does is dispatch closures of blocks of code on queues that are serviced by a dynamically adapting pool of threads. Without closures (blocks), you can't put anything in the work queue easily. Closures are what makes GCD a good idea.

[NT1440]

I don't get it, because its beneficial to apple it somehow takes away from them doing it?:confused:

When does a corporation ever release anything out the kindness of its heart?

[iSee]

This is a good move on Apple's part.

After all, how much money were they going to make licensing GCD, even if they wanted to be in that business? (Retorical question.)

This will encourage GCD development which means faster or more capable apps which makes OS X more desireable which means Apple sells more Macs (especially Mac Pros, I suspect), which is what Apple really wants.

Here's one scenario:

A 3D app developer has a 3D rendering component that is part of both the frontend workstation application and in a backend rendering service. The front end app might run on Mac OS, but the rendering service most likely runs on a server farm with some flavor or unix. Now the developer is more likely to port to GCD because both the frontend and backend apps can take advantage.... users win, developers win, Apple wins... nice :)

[aristotle]

I think a fair few oss advocates would disagree :P Apple's history is hardly great, with like two exceptions...

They're just using it to their own advantage as they have done many times in the past - any commits from apple to any oss is a 'positive' in some respects, little more.
http://www.opensource.apple.com/

I think you are confusing the general oss movement with the foss advocates like Richard Stallman. He has a unique definition of "freedom". :rolleyes:

Open Source software is divided into two camps. There are the OSS group which accept many licences including the GPL and BSD as valid choices and then there is the GNU zealot group who believe that the GPL and LGPL are the only true licenses that all software should be under. They like to preface their movement with "free" much like how communist dictatorships call themselves the "democratic republic of".

[MorphingDragon]

This is sad misinformation and unfortunately keeps alive a myth that undermines Apples efforts in this regard. Off the top of my head these are some to the technologies Apple has supported over the years:

gcc, especially the PPC improvements
llvm
clang
cups
webkit
X
ruby


This is just a quick list off the top of my head and thus I urge anybody with more accurate information to add to the list. I'm pretty sure I'm missing stuff. macosforge has a few projects listed there.

Certainly some stuff they do in a way that the open source community objects to. GCD dispatch is an example of a complete project coming to term and no open source involvement. While the open sourcers would prefer to be involved from the get go, it is also obvious that Apple doesn't have that luxury.

Maybe the open source community is upset that Apple doesn't sponsor every project. I'm not sure what the issue is but personally that would be the last thing I'd want to see in the open source arena, we just don't need one company dominating here. There is also the problem of some in the open source community demanding things be done their way or it is not acceptable, that is way to narrow minded and frankly I'm glad we have a plurality of open source licenses to choose from.

In any event I think Apple needs more balanced reporting on this issue.


Dave

They support Darwin :rolleyes::D *If you dont get it you FAIL

[Imhotep397]

In some respects I think this could be helpful, but as far as Pro applications and the data center market which will benefit the most from these technologies I'm not so sure. A lot of Pro media apps and server technologies were ported to or implemented on Linux in one form or another, before they were on the Mac so this could potentially help Linux extend it's "Benefits" list for those market segments more than help OS X. Conversely, I can see where this may be seen as necessary by Apple, because of the number of entrenched standards that don't use this technology would eventually choose a different set of similar tech and leave the Apple market with another "Legacy Technology" that may have been brilliant but never gained any traction outside of the small Mac market or more importantly outside of a small group of developers. I think the idea of not being quite as closed about some things has really been the well learned lesson that Apple has brought into the 21st century that they didn't have in the 20th century, so this is probably overall a better way to move forward.

[Spades]

Ars Technica has a great overview of Snow Leopard, including an excellent explanation of Grand Central and why it's a big deal.

http://arstechnica.com/apple/reviews/2009/08/mac-os-x-10-6.ars

(Spotted this link on Slashdot earlier today.)

The whole explanation starts at page 8.

This really is exciting stuff. If only my work wasn't .NET centric, or I had a good idea for a home project. I'd love to use the new things in Snow Leopard just for practice and fun.

[chris200x9]

I'm a windows to mac to linux convert (I'm weird :P) all I can say is thank you apple, now just let us ssh onto ipod touches without jail breaking! :p

[Eidorian]

This really is exciting stuff. If only my work wasn't .NET centric, or I had a good idea for a home project. I'd love to use the new things in Snow Leopard just for practice and fun.Don't feel to bad. (http://www.google.com/search?q=Concurrency+Runtime)

[After G]

I just hope potential adopters don't get "Not Invented Here" syndrome.

[Stratoukos]

Like all corporations, Apple does what's in the best interest of Apple.

Exactly. Even the idealists/altruists you are talking about (the individuals, not the organizations) are probably contributing to a project they like and they are using themselves.

This is the point of open source. Make better something you are using and release it to the community. RMS said so himself when Microsoft GPLed some linux drivers for their Hyper-V platform.

[twoodcc]

i think this sounds great! i'm all for more unix/linux/apple integration

[thither]

Well, not to discount Apple's work with webkit, but let's not forget that its origins were in a non-public fork of KHTML (Apple didn't have the best track record of working with the KHTML folks, either, see the wikipedia page for webkit for background).

In terms of this project, it's kind of cool technology but I find extremely unlikely that Linux / Unix types will embrace a non-standard extension to the C language which has been implemented by a corporate entity with an extremely mixed record on openness. Heck, Objective-C is probably one of the most "open" technologies Apple uses, but it's essentially not used anywhere but Apple, certainly not in the Linux world.

[shamino]

Uh... pretty sure adoption of non-standards is how a lot of great things become.. you know.. standard.
As a matter of fact, many standards bodies (including ANSI/ISO, I think) require commercial implementations of a technology as a prerequisite to standardization.
In terms of this project, it's kind of cool technology but I find extremely unlikely that Linux / Unix types will embrace a non-standard extension to the C language which has been implemented by a corporate entity with an extremely mixed record on openness.
You mean like C++ (which began life as an internal AT&T project)?

Don't forget that both C and C++ languages are still works in progress. Both are on the second version of their respective ANSI/ISO standards, and both languages still have active working groups that are developing new features.

I don't think it is hard to believe that a feature like blocks doesn't have a chance of being developed into a feature of the next version of these standards, regardless of what some people in the Linux community might think about Apple corporation.

[Xapplimatic]

My question to the code gurus becomes thus: Is there a way to make this work then for PowerPC OS X 10.5 Leopard users feeling left out of the 10.6 goody train? There are a lot of dual and quad core users feeling GCD envy..

[glossywhite]

Don't go confusing "open-source" with GPL code - there *IS* a difference. Also it irritates me when people call GNU/Linux "Linux". It's high time people started watching some videos, and learning a little *nix history - GNU & the GPL came a *long* time before Linus Torvalds wrote his kernel... YEARS before!.

If you're going to talk about free software, please TRY and term it correctly :p

[ChazUK]

Wirelessly posted (Mozilla/5.0 (Linux; U; Android 1.5; en-gb; HTC Hero Build/CUPCAKE) AppleWebKit/528.5+ (KHTML, like Gecko) Version/3.1.2 Mobile Safari/525.20.1)

Great news apple. I hope the open source community get a lot of use out of this.

[MWPULSE]

Wirelessly posted (Mozilla/5.0 (Linux; U; Android 1.5; en-gb; HTC Hero Build/CUPCAKE) AppleWebKit/528.5+ (KHTML, like Gecko) Version/3.1.2 Mobile Safari/525.20.1)

Great news apple. I hope the open source community get a lot of use out of this.

Same, n surely that is the main thing.. Apple have developed this brilliant API and technology that can be used to make computers faster and users less preoccupied with what they're computers are doing and more time worrying about the important things.

In a sense i guess Apple is spreading the love :-)

PTP