I wish someone could just tell me if this is happening to them, it's driving me insane. It was doing it on the GM release, I upgraded to official release and it seemed to be fixed, but I started watching a video a few days later and after pausing the video and unpausing it the problem came back. I would normally just avoid using Quicktime but there are certain websites with embedded video (like Keynotes) that utilize Quicktime and it does on there too.
I reported it as a bug to Apple, but I have no idea if this is a widespread problem on Mavericks since no one is saying anything.