I found 16GB very limiting and that has been all I've owned until this year. I usually had to delete stuff and manage my storage more than I like. I used iTunes match to store my music in the cloud and that helped, but still had to be careful of pictures and videos filling what was left after my apps. The jump to 64GB was VERY worth it for me.
In the end, this question is very user dependant and can't be answered 100%.