You should be able to hook your iMac to a 5.1 receiver via TOS-LINK (Optical Connection). You'll need to buy a mini-jack adapter in order to hook up optical through your audio-out port.
The audio-in and audio out port supports both analog and optical jacks. Not sure about 7.1 capabilities though... might have to hold your breath until Apple comes around to supporting HD audio via mini-display ports in their products.