In theory, the point of using the GPU for H.264 acceleration is that the GPU is more efficient at it than the CPU using less power and generating less heat. So if the cooling system in the MacBook Pro can handle the CPU doing H.264 decoding, it should be able to handle the GPU doing it. H.264 decoding is less strenuous than playing games and likely Core Image acceleration too, both of which Apple supports. Adding H.264 isn't likely to yield a huge jump in potential 8600M GT failures. And 8600M GT failure mechanism isn't heat itself, but heat cycling, so in theory keeping the GPU in use and warm at a stable temperature is better for it.
I'll repeat my earlier response to first-gen Mac Pro options.
http://www.barefeats.com/nehal05.html
Despite Apple not wanting to officially support the HD4870 with first-gen Mac Pros it actually does work since ATI always includes both 32-bit and 64-bit EFI ROMs. I believe 32-bit EFI is required for first-gen Mac Pros. Note this is not the same as OS level 64-bit support so shouldn't affect the first-gen Mac Pro's ability to run 64-bit Snow Leopard. nVidia skimps by only including 64-bit EFI probably to use a smaller ROM chip, meaning only later Mac Pros are supported.