IMO, there's two main costs involved. There's the tangible direct costs of making the changes to OS X's code & maintaining it, which is easy. But there's also the less tangible indirect costs of what I'll simplistically call "Control", which is hard. Simply put, the latter requires ceding to some of the DRM requirements, which places constraints on Apple's current & future freedom of OS evolution & features...and trying to figure out how to place a discrete, tangible cash value on this. That's why it isn't easy.
Can it really be that difficult? Apple ceded certain things to the license holders of DVD back in the day and they worked it out fine. I also thought mini-display port and hdmi on all the macs supports the DRM that Blu-Ray requires.
I am unfamiliar with the code, but I don't see why Apple would have to fork development of OSX to display a video. Just wrap it up in the Blu-Ray player.
Apple makes customers pay for MPE2 compression software separately. One either gets it bundled with an Apple product like FCS, or buys the codec from Apple as a $20 purchase.
http://store.apple.com/us/product/D2187Z/A
I think Apple could do this with very little hassle, and charge the people who want it a fee they would gladly pay.