You aren't imagining things re: negativity on this forum. There are many here who should have the term "hardcore whiners" tagged to their names. And it's basic human nature to loudly complain when something goes wrong, drowning out the VAST, VAST majority of users with little or no issues.
To answer your question about LR: Lightroom manages memory, and does a great job. There is only X amount that it can work with, so if you are displaying full resolution images, that memory will fill, especially when viewing at full screen.
Lightroom tries to "plan ahead", and begin fully rendering the images in the folder you are viewing -before- you get to them. But if you do it quickly, you will see that delay before the image "sharpens up".
I have a late 2012 iMac, full bells/whistles along with SSD. It renders a tad slower than the iMac Retina. Which is amazing, considering on the Retina I'm viewing has four times the pixels when compared to earlier iMacs.
Buyer's remorse is a natural occurrence. If it hits you, and you should happen to still have access to your older iMac, try this: After a day or two with the Retina, go back to the old machine. Look at some of those same images on the older machine. It's pretty shocking how flat they look, oft times boring looking. It will cure any doubts you had about this amazing machine.
----------
Not true.
Apple claims they do a laser calibration of every iMac that comes off the line. But you can still use your own.
That said... I just got the Retina, and (at least mine) is by far the most accurate iMac I've had, color profile wise, straight out of the box. I haven't had time yet to do a Spyder 4 calibration, but I don't imaging I'll see much difference.