Yosemite caused significant lag on my mid-2014 rMBP, and it was enough to cause me to revert to Mavericks. It was incredibly off-putting.
I would say that the whole system was perceptively slower, but I notice particularly egregious lag when I would open stacks/folders on my dock.
It's unfortunate, because I have an iMac 27" that's four years old and was running Yosemite pretty well. That said, it was still noticeably slower than Mavericks. There are other reasons why I reverted that particular machine to Mavericks, honestly I'm just not a fan of Yosemite, but the animations generally ran smoothly if not quite as smooth as Mavericks.
Surely it is an optimization issue; Mavericks on the rMBP is perceptively faster in every way than the iMac. Under Yosemite, though, the advantages of the iMac in terms of performance are nullified by the fact that it doesn't have a retina display and struggles to look crisp on Yosemite. It's Apple's planned obsolescence in action - pressuring consumers to get Retina displays. I'm particularly unimpressed with Yosemite and will run Mavericks until I'm sufficiently convinced to move to Yosemite - and it's going to take a lot of convincing because this is the first release that I have been particularly displeased with. I even liked Lion more than Yosemite.