I think that's it.But maybe I'm just saying that because I'm used to the speed of the iPhone 4S...
I have played with iOS 7 in an iPhone 4S too, and I can understand your point of view, of course
The hardware difference between iPhone 4 and iPhone 4S is noticeable: dual core Vs mono core, and a "9x faster" GPU (it is a lot more powerful indeed). That's why I say for me, is impressive the performance of iOS 7 in iPhone 4.