doesnt matter what program im using when it comes to sound.Is your Mac Mini connected to a TV and a Monitor, or just the TV? I had issues with some of my shows/movies stuttering when I had a 1080p Monitor hooked up through my Mini-Display and a 1080p 40" TV to the HDMI.
I always have issues with sound playing from PLEX if I leave it running. If I turn the TV on and start a Show/music it plays from my Mini's speaker. If I just restart PLEX it works fine.
If all else fails, a good uninstall/reinstall of PLEX could do it some good. Or the dreadful Format/Restore of OSX, from a clean start instead of a Time-machine backup.
Also, turn off automatic scan or whatever it is that makes the Library update constantly, the CPU load from Plex seems a bit taxing for the Mini's C2D.
only have 1 display connected.