Crossfire does not currently work in OS X. It would only work in Windows and possibly Linux.
ATI recommends CrossFiring only within the same family of GPUs (i.e. 2xxx, 3xxx, 4xxx, etc.). There is a crossfire chart available on their website somewhere that says which GPUs can be CrossFired together. The 3870/4870 combo does not work or is unsupported AFAIK.
Without doing the math, I suspect the power supply may be able to handle both configs you mention above. The biggest problem you would run into is that the 4870 needs two PCIe power connectors, and the 3870 needs one, for a total of three. The motherboard only has two power headers for the proprietary power cables so you will also need to run a Molex power extension from the optical bay.
But I ask, why would you want to run so many cards (unless you had like 5 or 6 monitors)?