See my review and benchmarks from a photography perspective here...
https://forums.macrumors.com/threads/1692536/
Short story: Aperture, Photoshop (and I'm assuming Lightroom, although I don't use it) are very demanding applications. Working with GB libraries of 30MB RAW photos can actually be more demanding than working with HD video. For optimal performance, you need high clocks, multiple cores, tons of RAM, and fast I/O. They also use the GPU for some functions as well, and that will certainly improve over time. The improvements in my workflow coming from a modest 2009 Mac Pro have been substantial.
What I determined through testing, and research is that the top turbo speed is not obtainable unless the system supports putting cores to sleep. Apple doesn't implement these C3/C6 sleep states on any of it's systems, nor do any PCs that I could find. In my research, the only instances where someone was able to get the top-rated turbo speed out of their CPU was by going into the bios for their system and manually disabling all but one core on the CPU. The fact that Apple and even PC vendors don't support this is probably a good thing, since the overhead in sleeping/waking cores would probably impact performance more than an extra 100MHz boost in clock speed would help... especially with the number of threads running at any given time.
You can read more about it in the thread below, but the moral of the story is, do not make any decisions based on the top-rated turbo speed. That is simply not obtainable. Best your system will do with lightly threaded tasks is one rating below max (3.7GHz for 6-core, 3.8GHz for 8-core, 4-core untested as of yet).
https://forums.macrumors.com/threads/1694931/
p.s. It has nothing to do with Xeon or non-Xeon processors. My rMBP behaves exactly the same way as do i7 processors.