So far the main differences, other than the lovely new interface are that Apple music is only available on the Apple TV 4 and of course, the AppStore.
That's the biggie as far as I'm concerned, before too long the AppStore will begin to fill up with awesome apps and games, if that floats your boat
Oh, and it's a personal preference, but I loooove the new Siri remote compared to the old one.
I think overall the single biggest difference is potential. The new Apple TV has got so much room to grow and the power to facilitate that growth.
But it all depends on your needs. If your happy with the selection of services on the Apple TV 3 and don't plan on using any more than that. You don't need the ATV 4.