iPod touch Odd Touch Behaviour - frequent recovery mode

Since I've had my touch at least 3 times a day when I try to turn it on it will be in "recovery mode". If I hold the two buttons down it comes on fine and runs with no problem. I've noticed no battery problems and it's not a jailbroken touch (it was at one point, but I reverted back). I've also done the restore dance and have had no luck.

Any help?