I had a very similar thought and experience with my iMac recently. This past summer I bought a top-of-the-line BTO 27" 2011 iMac that specs at 3.4GHz quad-core core-i7, 16GB RAM, and a 2GB AMD Radeon HD 6970M. You would think that this is one very powerful machine, and to be honest, it is. It is very powerful, when the software takes advantage of it.
With that said, the latest version of Photoshop that I own is CS2, that's right Photoshop CS2. The current shipping version is CS5 and I'm waiting until CS6 ships before I upgrade. CS2 was compiled for PPC while all versions after it are Intel only. I normally run OS X Lion (10.7.2) but when I need to use Photoshop, I have to boot into my Snow Leopard (10.6.8) partition. Here, CS2 runs in PPC emulation under Rosetta. And when I run CS2 it's slow because of two main reasons, 1) it was originally compiled for PPC, 2) it has to run under emulation on an Intel-based Mac. In fact, I went to see how CS2 sees my Intel-based Mac and it sees it as a PPC G4 with AltiVec clocked at 2.1GHz with 8 physical cores and 16GB of RAM but only has 3GB available (not that any of that matters as CS2 will only utilize one core as it isn't multi-core aware or 64-bit).
I first thought that a 2.1GHz G4 would be plenty fast for CS2, it is definitely faster than my old Mac (a real G4 machine with a single (physical) core clocked at 1.3GHz and 1.25GB of RAM). It was no surprise to me how slow CS2 would be running on my core-i7 iMac as I knew what I would be getting into. I was however, really surprised at how slow CS2 actually was when I was running a real-world process. Although it was much faster than my old Mac (about 3.5x faster according to the process I ran) it was much slower than I expected.
At first I was thinking that I would get significantly faster speeds but like you, I was actually disappointed at how slow it ran. Then I did some research and found out why things are the way they are. Unfortunately, there's nothing end users can do but hope that developers take advantage of the power in these machines.