I've never understood the obsession with IPS displays. Do you not adjust the display to be the best possible angle when you use a computer anyway? Having better viewing angles from the edges only really is improving the screen for those sitting next to you spying on what you are doing rather than yourself.
That being said, based on the OP's use case, I would HIGHLY recommend the rMB. I had the exact same dilemma as you, and I went with the rMB after playing with the two for a bit in the Apple Store. After moving to the retina display when the first rMBP was introduced, I went from "I'm only getting the computer for the SSD, the lightness and the power, I could care less about the display" to "holy crap, my eyes are bleeding because I'm not looking at a retina display."
On a smaller display, the retina display takes a lot of the strain off your eyes because you're looking at a crisp, clear picture rather than something that is pixelated and blurry. I really would have preferred everything about the MBA over the MB, minus the r. If the MBA was retina, it would be a no-brainer. But because you have to go rMB to get the retina, that's the direction I went. Retina supersedes ports, and I do a lot of photography and thus use SD cards and external hard drives frequently. Still prefer the rMB.