When you insert the DVD movie into your Mac, just let the DVD player program begin the movie (or jump thru all the movie trailers) but when the main feature movie begins to play, click on "Go" tab in the DVD Player menu and scroll down to "Title" and note what title number is playing at the beginning of your movie.
Close DVD Player and open Handbrake but close the window that Handbrake opens to scan the DVD and instead click on Handbrake "File" tab to click "Open Source (Title Specific)" and enter the title number that you observed when DVD Player began the movie. This eliminates the scruff that often loads along with many movies and your movie will rip to your hard drive just fine. I transfer all my movies this way for my AppleTV3 to play. Not for Blue Ray movies.