I just picked up a refurb iPod Touch 5, which has more or less the same hardware as the 4s, and set it up identically with apps, so it matches my iPhone 5s. In real world use, I can't tell any difference. Maybe if I think about it really really hard I can imagine that it might be a little less smooth when scrolling? Keep the 4s.