It works quite well actually. I had to replace my iP5 the other day due to a malfunctioning sleep/wake button and the genius bar assistant was floored at the superior condition my phone was in without a case.
My kids use Otterbox Defender series (at my direction). After a year's worth of use, the glass has marring around the headset speaker due to the exposed nature of the case in that location inviting tiny particles in.
Another useful feature of using the Oakley sleeve is when I need to set the phone down, I always lay the sleeve down first and then lay my iPhone on top of it. Therefore, my iPhone NEVER lays on a bare surface where particles could scratch the glass or back plate.
This might sound cumbersome, but I assure you it is super easy and protects the iPhone from scratches if you prefer to go "naked" like me but want to ensure the iPhone looks mint after a year's worth of use. You were talking about wanting to remove the iP5 from its case. If you don't have some sort of protection (completely naked), the screen will eventually look like ***** from months and months of abuse. The glass on the iPhone is susceptible to tiny hairline scratches from sand particles, zippers, etc. So if you go completely naked, I recommend some sort of screen protection. For me, the screen protectors look like crap after a couple months and I can't imagine using the phone with anything obtrusive on it. So I just slide my iPhone in and out of a silk sleeve and the screen stays in mint condition and clean of finger prints.