The thing I (and most consumers) like about TVs is that most of the time they're simple. You don't have to be bombarded with updates. It doesn't have to be connected to the internet. You set it up once and you're done for the next 20 years. That's why Samsungs, Panasonics, and Sonys work just fine.
Apple doesn't have a market here. It's not like a cell phone, tablet, or computer. The user/machine interface for a TV is kept a minimum versus those other products. On a TV you turn it on, and change the channel or input. Can Apple really improve the experience of changing the channel? I doubt it.
I think where Apple might have an Edge is with an app store. IF they go that route. A multi-touch control that would allow IOS games and things to run on the TV would be a BIG selling point. Apple is all Ecosystem all the time. I'm sure this set would have all the features of the Apple TV and then some.
But you're point about the internet is valid. Though, most people aren't without internet at home or wi-fi these days. So it's not a niche market either.