It really sounds like you...
... haven't quite given this thing a chance. I could be wrong, and you can feel free to ignore me. I'm quite used to it
Quite a few of the things you're asking for have been integrated into Spotlight, but either you haven't found them, or they aren't 'easy' enough.
I can't tweek how it works.
I'm not entirely sure what you mean, but you can add 'spotlight comments' to any file by simply going to 'get info.' That way, you can add words or phrases which effectively 'tweak how it works.' In fact, you can pretty easily add an Automator script that does this for you to the context menu in Finder.
I can't easily opt in or opt out drives or folders from the indexing process.
Again, maybe this isn't 'easy' enough, which I assume means the least-number-of-clicks, but you can add folders or drives to the Privacy tab in the Spotlight control panel. This can probably also be done in an Automator action.
I can't restrict it to any metadata subset should I not want to see the whole load.
Actually, you can use the kind: or date: keywords. You can also choose "Other..." when creating a Smart Folder. Go to
Apple's Spotlight Tips page and you get pretty useful information on how all this stuff works.
I hate the way it starts producing results before I am finished entering my query.
Here I can't help you, because that's a Jobs-ian feature of Spotlight, and it's very like him to not include an on/off switch for that. But I'm sure many Apple programmers would tell you that there's no point in having "instant" results if they aren't instantly updating.
In spite of what is said in the scant documentation, it won't search for phrases only - it just finds files where all the words in a phrase happen to reside - crap.
I believe much like a regular search engine, you can use quotation marks to denote a phrase. But it sounds like you've tried this and it didn't work right for you, and that is odd. It does work for me, so something on your system seems a bit glitchy.
Since finder find is now a Stoplight slave, I can no longer reliably find files based upon data ranges or file size. I can no longer use it to find hidden files or files in my System folder.
I'm sure there is a way to restore that behavior. Many people prefer that they don't get random UNIX /etc or Library files when they run arbitrary searches - you're not one of them. You've found a suitable tool for finding them when you know where you're looking, though, so good on 'ya. Another great tool for that is "find" in the shell of your choice.
Basically I think it is a toy. If they would let me restrict it to just music files say, or photos, then I might use it.
Weren't you just complaining that the searches weren't robust enough to include the system folders?

Sorry, cheap shot.
At any rate, sounds to me like you found a tool you're going to stick with anyway, which is great. I just thought I'd add this in case you didn't know some of the features you wanted... exist.
-rand()