Connect the technology dots
Here are some points to consider when speculating about the new Mac Pro design.
1) Heat -
The biggest issue with a tower design for both PC and Mac - heat build up and more importantly how to dissipate it. Excessive heat is the number one reason why computer chips fail.
The primary heat generators are the CPU(s), GPU(s), hard drives and power supply.
The Mac Pro when it first came out was "revolutionary" in heat dissipation because it channeled air flow separately for some of these heat sources and thereby cut down on the number of fans normally associated with towers of that size. Take a look at all the PC case options of this size and count how many fan mounts there are.
One of the "problems" with the XServe was it was described as hearing an airplane taking off when it turned on because of the noise generated by the fans since it didn't benefit from the same airflow technology of it's Mac Pro Tower sibling.
2) Thunderbolt 1.0 - 10 Gbs per channel bandwidth. Thunderbolt 2.0 - 20 Gbs bandwidth per channel.
3) SSD - 4 Gbs read ability. Raid 2 SSDs and you saturate (read limited by) Sata's 6 Gbs bus. No single hard drive comes near to this kind of data throughput.
4) Apple's Thunderbolt display POWERS a MacBook Pro. I know the Thunderbolt spec says it supplies 10W of power. I know my old aluminum MacBook Pro 17" has an 85W power supply.
5) Apple makes a fanless desktop 27" iMac.
6) Apple recently hired AMD graphics engineers
Assumption:
Apple makes their own specs for Thunderbolt (power)
My guess:
Apple's new Mac Pro attacks the heat build up problem by taking advantage of Thunderbolt. They move the storage and power supply outside of the motherboard chassis. They literally cut the heat generation in half in the sense of putting 2 heat sources in one chassis and the other 2 heat sources in a separate chassis connected via Thunderbolt.
From an engineering point of view, this would be a logical separation, too, as the CPU speed in accessing RAM and the video cards is dramatically higher than it is for accessing storage - the whole reason why there is a cache.
If my guess is true, it would make sense that no optical drive would be part of the CPU unit. It would be part of the Thunderbolt storage unit which I envision would be some kind removable drive bay system.
While it would be cool (pun intended) to house the PCI slots (video cards) separately, I would agree with the other responses that current Thunderbolt specs are too limited for that.
My speculative question is this - How big of a monitor would Apple have to make to house a couple of Xeon processors and a couple of video cards without fans? After all, what else could those new AMD graphic engineers be working on?
But in summary, the main reason I believe for modularity is about the heat. And the technologies that I highlighted point to a possible solution - CPU, memory and PCI slots in one housing connected via Thunderbolt to a Storage Bay and power supply in another housing.
1) Divide and conquer the heat issue.
2) Provide flexible purchasing options for more custom tailored "Pro" systems.
Do you need lots of storage? or a little storage? Do you need lots of PCI slots? or just the 1 video card? Every Pro has their own work flow needs.
One last thought -
Even if the new "CPU unit" needs to have it's own power supply, it would be significantly "smaller" than one that would power an entire existing tower. Think about Apple's external power supplies and of course lower wattage power supply sizes.
My 2 cents.