What are you trying to do? Handbrake will not output to video_ts, but there is a mactheripper alt that is free. I believe it is called FairMount. This basically will just mount your DVD and then you can copy the Video_TS folder to your desktop. I found it takes longer than mactheripper, but may help where mtr will not work.
DVD FAB was what I used on my PC. That was a AWESOME program.
On another note, now I am getting a random "VIDEO_TS folder cannot be compressed when I drag the file over to Toast to burn. Any idea what that could be?? This is really aggravating!!!