Common issue, no fix. It appears that there is some hand-shaking going on between AppleTV, TV and any sound system to get everything at the appropriate resolution, appropriate sound, etc. And it's those seconds where- instead of building in a start delay long enough for that to complete- the video itself is already running.
Conceptually, someone may have TV & audio systems that can do this handshaking stuff super fast? If so, perhaps they see little-to-no delay? But the usual experience is a black screen between format changes and then dropping in with the video already in progress.
You'll notice the same when Google inserts the commercials- there is a block of time to switch formats (usually black screen) before and after the commercial... often leading to your return to the video being skipped ahead by at least a few seconds.
Note that this is not just a Youtube problem. Any video you attempt to play that is a different format or has a different audio format than whatever you were just watching is likely to have this same effect... even video you're ripped from disc.
If me, I program AppleTV to have a short delay during format changes to try to alleviate this. But I'm not Apple.
User remedy: be ready with the back button to jump backwards a bit to get to see the missed bits when the video you want to see is running.