Assuming there is an intrusion (not necessarily malign) it should be either via remote control or LAN. The first thing you must do is to secure your router (change WiFi password for starting, and be sure there IS a password!). There are plenty of apps for macOS/iOS telling who's in your network.
Then I'll check if you have any extra remote control enabled in settings and disable it/them in case. I know, it's for infrared remotes so it shouldn't matter, but who knows. The Siri remote operates via Bluetooth and I'd believe there is a chance over a million (if that's even possible) that his/her remote (if any) is paired with your ATV4K. Still you may want to unpair all Bluetooth devices in settings (in there you should see anyway if there is a Bluetooth "stranger" enabled).