Your biggest problem is that you're encoding once to MPEG-4 with iMovie and encoding that once again to MPEG-2 or MPEG-4 AVC with Toast. Here's a couple of ways to potentially improve quality:
First way -
Output from iMovie using the native video codec (if it's an AVC camera, it will probably be AIC) and send the file to Toast, letting it do its own conversion to either MPEG-2 or MPEG-4 AVC. You might get better results by customizing the encoding settings (in the "Options" section under the Blu-ray Video pane - click the "More" button).
Second way -
Using these settings assume that you have outputted video that is compatible with BD specifications (it would have to be MPEG-2 or MPEG-4 AVC, no more than 26.0Mbps). I don't think iMovie can output MPEG-4 AVC (I could be wrong) and you won't be able to output anything as MPEG-2 without the MPEG-2 QT components (a $20 purchase from Apple).
In the Blu-ray Video pane, you'll see a blue section labeled "Options." Click the "More" button and then click the "Encoding" tab. Under "Reencoding, set it to "Never."
Try one of those methods out and see how it works out.