Cases eliminate the joy of the design for me. I'd rather have a few scratches after one year of use than mar the experience by covering it in cheap looking garbage.
There may be some perfect case that gives me the best of both worlds, and I do look, but never, ever find it.
This is an easy decision for the iPhone 4. No case.
Now the iPad, that's harder because the official case adds some features and fixes the one issue I have with iPad that... well I don't like living without even though I really really like the iPad design out of a case. I don't like how slipper it is on my lap, and I miss the little case-propper-upper-feature. Otherwise, I'd go caseless there too (just leave a slip case for it in my bag)
Definitely going caseless with iPhone 4 though, and so far not even the slightest sign of scratching even on the metal!