Partial solution
This is a reply I have had from Bob Jones (CyberTaz) on the Microsoft Community:-
Hello A---B:
A thread that you are following on Microsoft Community has received a reply:
Mac 2008 Excel Formula bar not docking
Reply
Again, not dismissing what you remember, but I believe you may have had the Formula Bar positioned to the left of the Formatting Toolbar. Prior to 2008 none of the toolbars docked within the document window... they were either snapped to the bottom edge of the Menu Bar or positioned elsewhere as floating objects. That would give the impression of it being one continuous bar.
I've never been a big fan of the 2008 UI but there are some things you may not have tried...
One is to right-click the Formatting Toolbar & select the Dock Toolbar in Window command to convert it to a floating bar. Then you can position the Formula Bar & the Formatting Toolbar side-by-side (i.e., end-to-end) in order to conserve vertical real estate. That should provide something similar to what you remember having had in the past.
Another option is to turn the Formatting Toolbar off altogether. The Formatting Palette of the Toolbox includes everything it provides as well as even more tools. The Toolbox floats & can be positioned anywhere you wish. It can also be collapsed/expanded by double-clicking its title bar so you don't have to keep turning it on/off.
If you're using OS X 10.6.8 or earlier there is an oval button in the upper right corner of the document window. You can use it to collapse/expand the toolbar area of the window. That button was taken away in OS X 10.7 so you may not have it. I don't know if something similar was restored in 10.8 because I haven't used that version of the OS.