Presumably another bug to add to the list... "just one software update away" 😉
I recall a version of macOS that in dark mode would present the keychain pages in black (black text on black page). That too was just a bug to be fixed to make the Keychain app address mode settings.
I think I recall seeing a setting in AppleTV menus to choose white black or auto. You might try switching that to something else and then checking chapter visibility again. Then, after tvOS updates, switch it back and check again. Eventually, the AppleTV team(???) may get around to noticing bugs and getting rid of them.