oh, just remembered. Target size mode is borked in snapshot 3 of the macgui which is why you got such hugely different bitrates.
I am sure 1180 kbps vs. 3068 kbps was no contest
Its fixed in current svn. http://trac.handbrake.fr/changeset/1865
Average Bitrate and Constant Quality work fine though. I suggest you compare using one of those.
User Note: Target Size is really not the best way to encode video, it encourages too much bitrate where its not needed, or way too little bitrate when you really need more. I actually pushed just to drop target size altogether but we decided to keep it and fix it.
Another Note: If you do use target size, realize that rate control on very short files has always been extremely tough. you will get bizarre results if under about 20 minutes of video. works okay on a full length dvd though. Again, whenever possible use ABR or Constant Quality.
ok then thats a massive help! thanks for that. i have never been a fan of constant quality because its too hard to get everything right, i tried it this time and its very good. thanks for the tip!