Just turn your WiFi off when you're not using your iPod. I do it out of force of habit ever since I got my 2nd gen. Pretty annoying if you're not used to doing it, and I'm not sure why Apple implemented it. Are you honestly going to be staring at your iPod for hours without using it, waiting for a push notification?
The only reason I can think of why Apple may have put this in is for radio apps. If you shut the screen off, WiFi will remain on and the app will still be able to stream music without the WiFi shutting down.
And it's not the iPod touch 4 specifically that's getting this "WiFi always on" thing, it's any device that updated to iOS4.