Make sure that you use only MP4 videos (.m4v and .mp4 extensions) which contain a H.264 video stream.
MediaInfo can show you this info. Some tools can copy the H.264 and AAC streams from MKV files to MP4 files.
The QuickTime X Player on Mac OS X 10.6.3 and newer can use the GPU (integrated or discrete) of your Mac for the hardware accelerated decoding of the H.264 video stream (from the file). QuickTime X does support this hardware acceleration only for certain file extensions (yeah, strange, i know).
AFAIK newer versions of VLC do support hardware accelerated H.264 decoding for MKV and MP4.