The only files I've found MPlayer can't play on OS X are Quicktime files and Quicktime and VLC do that. It also doesn't like Indeo Video but those codecs aren't available on OS X so none of the media players play them. I'd imagine if you are on an Intel Mac you should be able to compile MPlayer with the Windows binaries.