I also have that happen, but with the ClickToFlash plugin and the HTML5 playback. A simple reload of the website helps me.
Though I use that feature to my advantage, as I can open new videos in the same tab without needing to load them again, and I can just browse the comments, in order to find the bastards among them. Probably because I am one of them. 😱