My wife has the 3. I recently bought the 4.
You couldn't really tell the difference between them to be honest. I gave my 2 to my parents, and coming from the 2 to 4, it's pretty noticeable upgrade. Mainly just for the retina display if anything.
If money is a concern or if you plan on upgrading soon, I'd say just go with the 3. As I have a feeling the 5 will be more of a redesign to the form factor. Just my guess.
But, if you plan on keeping it for a while and don't care about having the latest/greatest and the extra ~100 isn't a big deal, I'd say go for the 4. Eventually the extra power will come in handy (io7 and greater), as well as having the lightning connector which eventually all of them will have