Up until the beginning of 2010, I owned an original iPod touch (got it as soon as it was released) and a dumb phone. At the time, it was missing a lot of the iPhone features: no camera, volume buttons, speaker, built-in apps, etc. in addition to lacking cellular data. Despite this, I was more than satisfied with it, especially when I had access to wi-fi (which in 2007 was quite rare--it got slightly better as time went on).
Nowadays, the iPod touch is more similar to the iPhone than it is different. The 5th gen touch is looking phenomenal, with the internals of the 4 and 4S combined with the larger screen of the 5. It even has text messaging! And wi-fi is much more prevalent especially if you're in a major city. It is possible to receive nearly all of the iPhone experience without paying a dime per month to use it. Another benefit is that you can save your phone battery by using it only for calling and use the iPod touch for everything else.
Ultimately its a bit less convenient, but I think sacrificing a bit of convenience is worth it if you can save a significant amount of money.