In general I've been ecstatic with the media aspect of it and not so happy with the PDA aspect. Now for the complaints:
The datebook is awkward to do any entry in, the audible alarms are too annoying to leave on and there are no silent (vibrate) alarms. It is quite difficult to put data in the phone book, and you can't search. The Notepad is useless for anything other than quick one liner notes to yourself as there is no syncing to the desktop, no cut & paste and no categories. As for to-do? There is no todo app, but lots of 3rd party ones you can use.
Biggest thing I miss though is Palm's system wide search. Nothing like it exists for the iPod touch. I use my Palm to keep work notes on so that I can do a quick search for a keyword while on the road. No such feature on the iPod Touch. You can search for music, but not for events, contacts, notes, to-dos or anything else useful for a PDA