Just real quick how would i go about doing this configuration? would i need that mini 6 pin to regular dual 6 pin running to the 5870 and just the other mini 6 pin to the 5770? also is it safe with the tdp?
I operated an Apple 5770 and a factory overclocked 5870 GB Sapphire Toxic 2GB without issue for 4 years including bootcamp gaming (inlc Crysis 2) with the 5870 using a Mac Pro 2008 Mac Pro. I re-routed power from the spare molex connector in the drive bay to form an 8 pin connector. So thats 6 pin for 5770, 6 pin for 5870 plus the DIY 8 pin.
In Windows, the 5870 drove the screen. In OSX only the 5770 was operating. The 5770 became frightfully hot though in Windows for some reason despite being "inactive". For powering monitors I can't see any issue. If you were to simultaneously max out both cards at once though perhaps there would be trouble.
For just powering the screen, I can't see any problem. However, technically it may draw up to 200% power from a single 6 pin cable.
If you want to use splitter. Connect the 1st 6 pin to the 5870, and then split another one to the 5770 and 5870.
In this case, if the 5770 only power the screen. It won't use too much power. And when the 5870 doing the heavy duty, it still drawing the power from both 6pin sources.
E.g. The 5870 really draws 75W from one of the 6pin (usually it won't happen unless your workflow is running Furmark). And the 5770 only use 20W for powering the extra screens. It only draws 95W from a single 6pin. Still technically over the limit, but should stay within normal buffer zone (assume 50% buffer for circuit design). And the Mac Pro won't shut itself down. This is a better option then connect both 6pin on the 5870 to a single source, which may draw up to 150W when under load.