I actually had this exact dilemma today when choosing what to get. Eventually I went for the Mac Pro;
2.8Ghz Quad-Core Xeon
6Gb Ram
2Tb HDD
5870
Expandability - If for some reason I wanted to;
Add more RAM
Add a new/more HDDs
Would like to add an SSD
Add/Replace with better GFX card
..then I don't have to sell the entire machine, I can just upgrade it.
If the HDD fails, I can replace it.
I was a little concerned about the possibility of the iMac getting knocked over, which was probably unwarranted, but the Mac Pro would be on the floor, so that's not so easy to move.
Also, this way, you can get a matte screen!
If you compare the two, the iMac initially looks like the obvious choice;
2.93Ghz i7 - fater than the 2.8Ghz Xeon
8Gb RAM @ 1333Mhz - cheaper than the 8Gb RAM for the Mac Pro, which also only runs at 1066Mhz
The SSD option for the Mac Pro is almost daylight robbery too.
And all of this, for less money!
--
It depends what you want it for really; I personally like to use gaming as a benchmark; though I don't game much myself, it's one of the most demanding things you can do, so a machine that's well-rounded enough should be fine, and to me, the iMac, whilst a beautiful machine, is a big MacBook Pro.
What I will actually be using it for is Music Production and Visuals in VDMX, which is CPU and GPU intensive respectively, so this machine personally would suit me better.