The iPad 4 is noticeably faster than the iPad 3. If you do not notice the difference then good for you. That just means that you lack the capability to notice it. However, that doesn't mean that there is no difference, because there is.
Hahaha. You lack the capability to notice it = it is faster, but not enough that it matters. For the stuff I do most of the time, it is not noticeable (by my incapable brain), but I do not play games or generally use apps that push the limits of the device. I use Safari, iAnnotate, Evernote, etc. I bet your use case is the most important factor. If you gave my parents a netbook, it would just as fast for them as a Macbook Pro, because they are just checking email. But, if you gave a netbook to a graphic artist, they would be writhing around on the floor in pain. So, the improvement is in the eye of the beholder. Not incapability, per se, but use case.
The battery life mentioned above would be great. And, I imagine that if I scrolled through enough of my PDFs side by side I could see a difference between the 3 and 4. What I hope is that someday Apple will come out with something so powerful that I no longer have to see those checkerboards on my iPad while I wait for a PDF page to render. Maybe the iPad 5.