I think you're right. I may try the Speck StyleFolio. It has a 30 day guarantee, so I can just try it out. I really want port protectors though. I read somewhere that one of the most common things to go wrong on iOS devices are the ports not working due to dirt/grim build up.
I've owned every model of iPhone and every iPad except for the iPad 4 and I've never had that problem or heard of anyone else having it. As long as you look after your device and don't intensionally do anything to allow first and other stuff to get into them then you should have no problem.