When I'm feeling a bit paranoid about these things, I do the reset, then use the device for a bit, and then do the reset again. I'm not sure if you can do that with Apple TV without having to put in another Apple ID though. But maybe you could make a dummy ID.
The goal is to write over a bit of the memory and then restore again. At least this is what I do with phones before selling them. But there is more vital stuff on those phones than on the Apple TV. So caution makes more sense.