The link to TotalFinder above looks interesting, I'll try that when I get home.
The bottom line is that this seems to stem from the Unix filesystem roots, where folders and files are treated identically. Of course, they have the option to "ls --group-directories-first".
I think Apple really should make it a user choice. I can see the argument either way.