It is more than speed increases. Certain workloads are going to get better improvements. And the GPU is moot for integrated graphics for the Mac Pro class machines.
Gets done to the low level specifics.
http://www.realworldtech.com/page.cfm?ArticleID=RWT091810191937
short summarize.
a. better decode of instructions. faster and low power ( the latter should allow turbo to turn on in more situations .)
b. more memory lanes. Will go from 3 to 4. if going to have 6-8 cores you need more paths to smaller chunks of memory. [ and it that just works for many old programs too. Maybe not 100% optimized but it does work, despite the naysayers around here. ]
#1 benefit of going to 4 lanes is that folks can stop whining about Apple having 4 slots in these forums. Sandy Bridge is oriented to 4 slots specifically ( yeah also 8 and 12 .... but there is no room with current design constraints for either one of those. )
c. there is a better bus between the cores. ( workloads which fork/join work will be happier. )
The wikipedia article on Sandy Bridge was a bit above my level of understanding.
Beyond the tech the reality is that Mac Pro is really on a two year major revision cycle. Every two years Intel changes the socket. That forces a redsign of the logic boards. That brings in new stuff. In the second year you mainly will get a speed bump. ( a slightly better processor and a few other bumped components. No new sockets/connectors. No major internal changes. ).
With a Mac Pro you buying more than just a CPU ( although that seems to be what folks fixate on in the forums. )