I try to be very discreet about using my iPad in public.
One, because they're still new devices and they do attract attention. I was recently in a dramatic production and while waiting backstage for my part I would use my iPad. Virtually every other cast member around was doing exactly the same: reading with their iPhones, BlackBerries, playing games with their PSPs, typing on their laptops, whatever. Yet I overheard some nasty comments about "that guy" who's "waving the iPad around". Just by virtue of it being new, everyone noticed the iPad while the phones and laptops remained invisible. (It actually left a real bad taste in my mouth about these people who I thought were my friends, but that's another story.)
Two, because I have been known to set it down and forget it. It's just too big to pocket and I don't carry a backpack or anything. Once I set it down near my legs at a restaurant and forgot all about it -- raced back for it 20 minutes after I left. Thankfully since nobody had noticed me bringing it in, it was still exactly where I left it, under the table I sat at.
Actually, for both these same reasons, I am strongly considering my next device to be an iPhone 5 instead of an iPad 3, when those respective devices get released.