What version of Handbfake are you using? I have the latest Mac version and there's no such thing as an ATV4 preset. There are several "Apple" presets at various resolutions, all of which the ATV3 should be able to play.
Assuming you mean the "Apple" profiles under "Devices": They should work up to "1080p30". The "1080p60" profile allows H.264 level 4.2, which is not supported on the ATV3 (up to 4.0 only). This doesn't necessarily mean that a given video won't work, but Handbrake may produce video that an ATV3 can't handle under this setting.