Do some research using the keyword "coreaudio" and "DTS-HD" for Macs.
Assuming you are ripping the media sources (blah blah legally of course blah blah)? You aiming for 7.1 or 5.1?
I can understand the
want for DTS-HD passthrough however I feel its highly overrated unless you are at the top of the audiophile food chain using some very high end equipment. Or you have a very specific technical reason. So its hard for me to justify the
need for it.
Plex offers you all the passthrough options you want (where applicable, DTS HD may not be there for MacOS software) to experiment until your heart is content. I do think Plex is by far your best bet though, it offers everything you could want with a massive support community behind it.
However, what you'll need to use (without a bit of effort, more on this) is Plex software decoding the audio. The LPCM should be lossless. The pros for doing this generally outweigh the cons because you can get the ability for display syncing, volume control (outside the receiver), compatibility across all media sources and hardware, and I'm sure some other audio manipulation settings since the software can see the audio. Conversely the receiver receiving a LPCM source still has the vast majority (all?) of its functions available.
Cons are the obvious, you won't get the little DTS-HD/TrueHD light on the receiver (since its already decoded). And since I know someone will say something, in
theory you won't get as good of audio quality because the decoder in your receiver
may do a better job. I'll add though, even if this is the case it would be incredibly unlikely you can hear this difference without background in audio engineering or something. Keep in mind the audio data its receiving is lossless, the decoding is just happening at another point in the chain.
The bit of effort I mentioned. I believe coreaudio is MacOS's bottleneck for DTS-HD/TrueHD passthrough. Using Bootcamp and even Parallels (WASAPI enable, bitstream disabled) can probably circumvent this from what I've read online but I'm not sure.
I haven't tried doing any of this since Yosemite but I can't find anything indicating Sierra or even High Sierra will be different.
If you are willing to drop the Mac from the chain I would probably just recommend a Kaby Lake Intel NUC. Then use Plex or Kodi. This would get you a native 4k decoding, access to 4k streaming services (HDCP permitting), HD audio passthrough, no lag or buffering, etc etc. I've been impatiently waiting for a new Mac Mini for this reason. But I'd settle for a 4k AppleTV.
