The CPU used in the 3GS lacks L2 Cache, the A4 used in the 4 has 512kB. Lets not forget about the clock speed differences as well 3GS 600MHz, 4 800MHz.I believe the iPhone 4 uses the same graphics processor as in the 3GS... but with the retina display has twice as many pixels to deal with. Because of that, some apps that rely heavily on the GPU can run faster on the 3GS than on an iPhone 4. The "choppy UI" you mention is probably related to that as well.
The iPhone 4S uses a much faster GPU though, and presumably the iPhone 5 will use the same or better.