Lovely idea. But as has been said many times before, if Apple is going to deliver the whole software experience in a cheap box, why buy the full TV? Apple's television is going to be made by someone else who will be very likely to take the exact same panel and put it in a less pretty frame with their logo on it and probably offer it for substantially less than the one with the Apple logo on it. It will be the exact same panel, which means the exact same quality of picture. If we can save many hundreds on THAT TV and then attach the full Apple software experience via a cheap add-on box, why buy the whole cow?
...Key to Apple's success is the lock of software to hardware. If they can be separately purchased, then the Apple Television hardware would be unique in that any head-to-head against other Television hardware specs would carry much more weight.
I agree that they would need to differentiate. I do think that if Apple were to produce an integrated display, it could be unique in several ways:
-beautiful design - more drool-worthy than the set-top box
-display quality - a thin, high-quality display with dead simple color accuracy, super sharpness, generous viewing angles, etc.
-cleaner setup
-user interface - touch? voice? gestures? easy input switching? something that isn't possible on the set-top box.
-greater performance/features. Faster processor (looser heat parameters), integrated FaceTime camera, decent compact speakers, more I/O
Really, it's why people spend $1,299 for an iMac when they could get a Mac Mini for $599. Both run OS X. One box, one machine, one cable.

People have multiple TVs, after all. Pick up the set-top box for your big Samsung. Then in a few years, move the set-top box into the bedroom TV and pickup the Big Daddy Apple display. I really do think there's room for both form factors.