I used to have a 3GS 16GB which was all filled up. However I had lots of high-res photos and a lot of music on it which I never listened to. I contemplated getting a 32GB 4S, but honestly I found no reason for it.
After having used Spotify for almost 2 years, I've realized I no longer need iTunes, or the iPod-player in my iPhone. I now have all of my music in the cloud and I can access it from any computer or any iPhone/Android et.c. Also, I do not sync my photos from iPhoto, and neither do I watch movies on my iPhone - except for Youtube. Thus, I only use my iPhone built-in memory for pictures taken w/ the iPhone itself, Apps, Podcasts, Books. For this reason, 16GB is perfectly fine for me.
For an iPad on the other hand, which is used for media-consumption, I would never get anything less than the 32GB model.