TVs to date are limited to 30fps. These new iPhones can shoot at higher fps rates. You could run them through a tool like FCPX to convert them to 30fps and still keep the slo-mo effect. Else, this is one of the more tangible things pointing toward an TV refresh (hopefully soon). I'd love an TV that can handle native rates of 24, 30 and 60fps (but more if it could do it).
I had the same problem. I don't understand why the playback software can't downsample the full speed video to 30 fps (cut out 7/8 frames) and then use them all during playback. Maybe it's just such a limited-use feature that it wasn't worth the workaround.
Key thing is AppleTV needs new hardware. This series of Handoff features begs for a new one with at least a microphone. Honestly what I might develop is an Xbox One-like system that is the main hub to the TV. HDMI runs into TV from AppleTV. Then a larger AppleTV box has either another HDMI input for a signal from an A/V receiver or just four or six HDMI inputs to work as a semi-receiver.
Overhaul the OS, and your AppleTV is basically the brain of your setup. Run all the various apps, or switch to old school stuff like cable/satellite that runs kind of like a virtual machine. That way AppleTV is calling the shots and could interrupt your TV watching if a phone call comes in. This means you can also setup overlays like stock tickers or ESPN scores.
It would require some horsepower since it would definitely not be any sort of hobby product. But I would be on that, especially with an App Store and third parties opening up their own subscriptions. You get HBO and ESPN to offer their packages without a cable company commitment and watch the money flow.