I don't use Airplay, but yes, air connected devices play compressed audio formats.
By the other hand, your amp seems to be a serious machine (reading a spec sheet). You could even get a better sound than your iMac default quality.
I read your amp is able to decode Hi-Res audio files. iMac has a nice internal DAC, but I suspect if you get your audio signal through Thunderbolt3 (digital) to the Yamaha, the analogue sound will be produced by amp's internal DAC, which looks a nice one.
Even using MP3 or AAC files, an external quality DAC (digital-to-analogue decoder) can easily give you a better sound than using bluetooth or airPlay.
You could get analogue sound from audio-jack in iMac wired to your amp too, instead of taking digital from mac and letting the Yamaha make the conversion to analogue.
You can try; I bet you'll get nicer sound through Thunderbolt (and MidiSetUpUtility should be used to select RXA870 as sound device).
All this is useless if your speakers are not good enough to tell the difference. And, of course, depends on your perception and taste also.