I just pick the lap top up, then tilt the plug down, and then gently remove it.
I'm pretty anal with my stuff, so I'm kind of careful but since it's metal to metal, over the time I'm sure very small nicks will pop up in there.
I take it by thumb and index finger and pull it out.
On my old MacBook Pro I ended up with a slight dent on the top side of mine from the Mag safe coming in and out so I'm trying to be more careful on my new machine.
----------
On a side note, Sean - how do you get on going from your rMBP to iPad Mini? I'm worried after using the retina display and iPhone screen that the Mini's won't cut the mustard...