I don't think the lack of tactile operability will hinder the use of the iPhone/touch being used as a controller. You will be able to play board games, like scrabble, and you'll have the board on your tv & the letters on your phone. Or play tx holdem and have the community cards on your tv and your hand on your phone. This could be done with people in the same room who have iPhones or with anyone else who has an iTV. This will become a big part of Apples xbox live type thing they are releasing later this year.
Also, you could play racing games and use the iPhone as a steering wheel. There's lots of games that could take advantage of a touchscreen controller that has accelerometer/gyroscope/etc
but think about how the iphone is currently used as a controller.. you're looking at the controls - without looking down at the controller i dont think it would work for more active games - games like scrabble yeah the iphone would be perfect for.
i want a basic bluetooth controller - like a redesigned NES controller with a couple more buttons.