The tvOS YouTube app currently doesn't support these features, unfortunately - so what you're experiencing is most likely a mismatch between the FPS of the video files and the refresh rate of the display.
Note: This is no longer an issue if you use an HDMI 2.1 display and a video playing device that properly supports HDMI 2.1 due to a feature called VRR. It is unkown if Apple is going to implement HDMI 2.1 correctly for the ATV 4K 2021 at some point in the future, it's currently not using any HDMI 2.1 video features at all.
