Hmm.. I have 4 AppleTVs. 1 first gen, 2 second gen and 1 3rd gen and have no issues with them at all. The second and third generation have no issues with Netflix at all, they all work flawlessly. I watch A LOT of Netflix, so I definitely would notice if I would have any buffering/low quality issues.
As far as the connection, I have VZ FiOS and have a sustained speed of 25/25MBps. One of the second gen AppleTVs are connected via the ethernet straight from the router itself. The other second gen and third are connected wirelessly. The third gen is a bit different in that it's connected via the ethernet but through a wireless extender (?) of the Airport Express. Anyway, I have NO ISSUES at all. I also have a PS3 and BD players and TVs with Netflix capabilities and they do not come close, in my opinion, to the AppleTV's ease of use and especially for my needs.