so i almost returned my imac because of this lag issue (switching between screens and apps is a big deal for me, no matter how fast everything is)... but i tried a fix that helped users on older imacs (non-retina and retina mbp) and sofar its working splendidly.
for the record, i have a base imac retina (i5, 8gb memory, 2gb video, 1tb fusion) and i was experiencing crazy mission control lag, and lag whenever my wallpaper changed.
so the fix was to download Onyx, and do a system and user cache delete, and did a restart. It was still buggy at first, but after a few minutes it's buttery smooth. Also, after a few hours with many programs open (including safari and some video programs), 3 desktops, it still works pretty smooth with a hiccup here and there but very rare.
i don't know why this works, especially from a clean new install from the store, but it did. im thinking the install leaves some stuff behind that has to be manually deleted, but thats just a guess.
so all in all now i'm pretty happy with this mac, and glad i don't have to upgrade and spend more money on the highest spec one.
hope this helps other people with the i5 and 290x.