The first week was rough, testing HD content from Netflix (30 Rock, Arrested Development, etc.) resulted in blocky, under-saturated video, which was way worse that what I was used to with X360. Apparently, this was some kind of settings issue between Apple and Netflix because within days this cleared up. I'd say content looks better now on the ATV than the X360. I haven't done a proper side-by-side test but I have done a one and then the other and I think it looks better. We don't currently have surround sound, just two little tv speakers, and I can't tell a difference in sound quality.
We've had the sound dropout issue, it appears to me to occur when the encoding had gone to a lower level (adjusted down for bandwidth reasons, what have you) and then come back up to the previous level (full HD stream, no blockiness, etc). As others have stated rewinding or sometimes even pausing and playing after a second fix this. It appears to be present even in the newest firmware (based on 4.2.1 I believe.) The new firmware also seems to have improved the speed with which streams start, however it doesn't appear to have improved the speed with which I can access Netflix from the main menu and browse Instant Queue, Recently Watched, etc.
A few gripes and bugs, specifically for Netflix: Backing up a menu will always bring you to the top of the list, regardless of where you where, as aced411 mentioned. Sometimes the thumbnails for the Instant Queue and other pages will take a long time to populate even if you've just been browsing them recently. The quality seems to waver quite a bit, and doesn't seem to be as dependent on the local network connection (or at least, your WAN connection) as say, the X360, which presents the audio bug mentioned above. All in all though, the improvements to this interface over the X360 make it better in the long run. I like that you can search and queue from the interface without having to go out to the web on another device, even drilling down to search by actor or director. Apple's update of the Remote app turns any iOS device into a handy keyboard and touch based remote, for this reason it's turned my old first gen phone into a useful device again for the living room. Browsing a full queue (as Netflix seems to automatically add stream-capable movies to your Instant Queue when you add their disc counterparts to your main queue) is much easier in the 'screen full of thumbnails' view of the ATV in comparison to the X360's 'long row of pictures'.
All in all, I'd say I'm happy with the ATV, if even purely as a Netflix streamer.
*note, I haven't tried any other major Netflix stream devices currently on the market like the Roku or Boxee offerings, my comparison is strictly based on the Xbox 360 Netflix app, which requires you to have a paid Gold account in good standing. As I rarely game any more, paying Microsoft upwards of $7 a month for the 'privilege' of streaming content that belonged to someone else when I was already paying Netflix seemed outrageous, hence my purchase of an ATV.