Solution
I've found a solution to my bluetooth reception woes that may interest other Mac Pro owners.
It involves purchasing a
cheap bluetooth antenna off ebay and running it external to the case.
I purchased both the one above and
this Apple part. Perhaps not surprisingly, the Apple part, sucks! Don't buy it. Even external to the case, it offers no improvement over the stock antenna. The one linked above seems much better.
You can use Apple's Bluetooth Explorer to measure the connection quality. Install Xcode from your OSX CD (options) and look for it in Developer>Applications>Utilities>Bluetooth.
Here's the performance of the stock antenna with the Mac Pro behind my desk. The first third of the graph is with the mouse furthest from the Mac Pro, the middle third shows the signal quality with the mouse nearest the Mac Pro and the last third shows the signal quality with the mouse where I would normally use it (about mid way between the two extremes).
With the mouse furthest from the Mac Pro, lag and jerky movement were the norm and this is supported by the consistent RSSI near -10 and the raw RSSI near -70.
Now here's the performance with the new antenna resting on top of my desk (again the first third is furthest from the antenna, middle third closest, and final third between the two extremes).
Note how it doesn't really matter where the mouse is on the desk, it's enjoying a great lag free connection with only occasional small drops in RSSI and consistent raw RSSI between -50 and -60.
Here's some pics of the antenna installed. The stock bluetooth antenna wire is visible in the first photo as it's labelled. Unfortunately, the connector on the replacement antenna is just too big to fit through the cheese grater holes on the rear of the chassis so I had to have it exit through a PCI slot cover as shown. The antenna wire is long enough to reach the top of your work surface and the antenna itself is protected by a piece of cardboard which makes it easy to tape to anything close by. While it's crude, and I wish Apple would have provided a better solution, this is better than being forced to use a wired mouse.
Of course, you could go the route of a 3rd party dongle, but then you forgo the ability to wake the Mac from sleep with your BT mouse/keyboard.