Fast-forwarding html5 video when switching tabs

I'm trying to run OS X (10.8.1) without flash.

So I have several tabs open (Safari). I start a html5 video. For some time I go to another tab. When I come back to the html5 video tab, the video starts fast-forwarding from where it was when I last left the tab. So it seems like when I switch tabs, the audio plays fine but the video stops until I go back to the tab and then the video fast-forward until it comes to the time where the audio is. This is of course not how I want it to be, I want the video to run in the background as you get with e.g. flash.

I hope someone understand what I mean.

Anyone get the same behaviour? Anyone knows why I get this behaviour? Anyone knows if I can fix it?

Thanks

edit: it was videos on twit.tv that I have tested on.