Try a restore? If your feeling up to it, break out the solder gun and try to replace the battery.
https://www.ifixit.com/Guide/iPod+Touch+4th+Generation+Battery+Replacement/9829
You can probably find a battery online for less then $15, I got a battery replacement for an iPhone for that was $5, works perfectly. I really wish Apple made the iPod touch 4 so you didn't need to solder it in and desolder to get it out
The iPhone 4's battery was a breeze to replace compared to what the iPod looks like.