Add this string to you advance settings within the ATV preset, using handbreak:
cabac=0:ref=2:me=umh:b-adapt=2:weightb=0:trellis=0:weightp=0:vbv-maxrate=9500:vbv-bufsize=9500
My HD/Blu-Ray films are spot on with these settings on my ATV.
I have surrogate, staten island, star trek, 2012, ice age 3, invention of lying, rambo, fight club etc..... all in HD and all play perfectly fine!!
Regards
C