Apple TV+ allows offline viewing, otherwise you wouldn’t be able to download them.
I couldn’t play downloaded episodes of Black bird when I plugged my Mac into my TV. This turned out to be because my TV was too old and couldn’t play the high quality downloaded file (HDR/Dolby vision or whatever it was).
Are you trying to view them on an external display? If so, check the download settings within the Apple TV app.