I agree. I had the ATV2 on my wifi and was trying to watch vids sitting on my NAS, or through the Mac Mini via the AirPlay mirroring. It would always drop the connection at some point either way. I was using the 5Ghz band.
I then added an Ethernet port into my living room and connected the ATV to my wired connection, and it was magic. No more lag. The AirPlay works awesome. Even my iPad and iPhone are always seeing the ATV2 now that it is wired.
Make sure you have a Router with Gigabit ports and use Cat5e, not Cat5 cables.