I have 2 ATV3s and 1 ATV4. The ATV 4 has these core benefits in my opinion:
Apple Music - Radio, Browse, Library, etc. They took this off the ATV3 so if you pay for Apple Music, you'll want the ATV4.
Other music apps - if you have other music subscriptions, or maintain your own library, the ATV4 has the most options with having an App Store
Plex - handy app if you use it (or pay for the premium services)
PlayStation Vue (or similar) - Most of those live-tv services like PS Vue, Sling or DirecTV Now have apps for ATV4
AMC, Comedy Central, TNT, Crunchyroll, Crackle, etc - Basically every channel or service that has an app is already on the app store.
The Remote - as much as I like the old silver remote, the new remote has a few features I like - especially volume control. It's also rechargeable (iOS/lightning).
Faster WiFi - 802.11n vs 802.11ac (MIMO)
Bluetooth 4.0 - Bluetooth speakers and headsets
Home Sharing - I suspect a later version of iTunes will no longer support Home Sharing (or Apple will move it to the TV App and drop LAN streaming support of the ATV3 complete). Currently, the beta version of iTunes I'm using supports ATV4 but not ATV3.
No optical audio out
USB-C for management/deployment
I also like the MLB.tv app on the ATV4 more than the ATV3. I haven't checked if its still supported this season though.