I have had a 3G iPod and a 1GB Shuffle, I currently have a 4GB Nano (with pics, calendar, and contacts).
My Library is 5GB & change (whittled down from the 12 GB or so when I did a mass conversion of EVERYTHING), and a lot of it is stuff I really don't want to carry - some spoken stuff, a couple of videos & TV shows, etc., so I set up a Smart Folder to prune out the things that won't ever go on the iPod, regardless of model.
I'll be moving to an 8GB iPhone in June, and think that'll suffuce using my current rules for quite a while.