I looked into this in great detail with my MM Server when I first bought it and had the exact same view as you. I spent hours talking to the Apple techs on the same question.
You can install standard OSX non-server version only if you drop in the image from another non-server MM this worked for me) or you install NOT through internet recovery. Simply put, Internet Recovery which is the standard way of installing OSX will automatically load the Server App. You don't get a choice if you choose the formal Apple/OSX option.
You also can't rip out the App unless you want to take some risks. I believe the middle (Apple Tech suggested this) ground is to untick "dedicate system resources" in the App so it is then very similar to the Network App in OSX. THe Server App still fully works, it is just competes with any other App for system resources (which is no competition in the MM Server as there are ample resources). You can still use the network app as your options menu as well instead of (or as well as) the Server app as well if you want to completely dismiss Server. From my research the "dedicated" option just diverts a portion of the system hardware resources (CPU/RAM/Ethernet Traffic) to the Server App. I am not sure what percentage is diverted but I have read that unless the MM Server is set to work as a dedicated server you would not tick the option. Makes sense to me. You can also look at it this way, if you do tick Dedicate System Resources then a portion of the resources will be permanently dedicated to Server tasks when you boot (almost like permanently running the App and its services). But if it is unticked then it is no different to any other App. That is, does Adobe CS6 or Parallels use massive resources when the App is not open? No. So just "de-dedicate" and make sure it does not start up a login and you always have it there as an option. Remember, once you have the operation for the sex change you can't just sew it back on! Yes my analogy is not perfect because you can also "re-install" the OS to fix the sex change
Having said all that, I don't tick the 'dedicate system resources" part as I suggested but still use the Server App now for file sharing and screen sharing (so I am glad i didn't have the sex change because I have begun to use it a little).
You also do input some server details (name/password of the server) when you set up OSX but that is just for access.
I would just leave the App (once you have initially run OSX install and input the Serve name/password) and not use it and untick the dedicated part and that will work fine. You can then always use it if you want.
You can google the "midway" disable server method I suggest, or you can also find the "rip the heart out of the chest" method as well. The second method tries to "delete the Server App" and just requires a few more "and search for this file in library... and delete it" type operations.
I use my MM 99.9% as a media station and HD sharing device for my home network and it works really well with the quad core. It sits nicely beside the flat screen TV and plays all my Vids (HD on a Sony Bravia). The Server App will not even test a minor percentage of the CPU/RAM under this operation if you don't use it and it is not dedicated. Video transcoding with Handbrake etc is also a breeze.
Anyway, that is my two cents worth. And yes, I will be paying the extra $20 for the Mountain Lion Server App (on top of the $20 for OSX ML) when it comes out.
Silver.
----------
Ah, cool thanks - just concerned about a few stories Ive read in regards to the server version using up a whole lot more RAM then the standard version - even with the server features off......
I am not sure on the new GPU obviously on the new MM but that comment on RAM in the current model is a little correct. The Graphics Card in the current MM Server uses the Physical Ram (not its own dedicated GPU RAM) but it is only 512mb if you have 8GB Ram and 384mb if you have 4GB Ram.
If you untick "dedicate systems resources" you have apple RAM with 8GB.
The MM Server has good graphics form my videos but if you are a Gamer/graphic artist I would do the research on the GPU. Remember is was designed as a server so the monitor is asleep 99.9% of the day so the GPU was an afterthought.
----------
Don't be concerned about stories and uninformed mass hysteria. As long as you don't enable any of the server's services, it cannot use more memory than an OSX client. I would just double check because I think the only thing enabled by default might be screen sharing and I am not by my mini to confirm right now. You can switch that off, if you really want, but the amount of memory it uses to run in the background is not something worth worrying about and I find it damn useful to do stuff on my mini whilst I am downstairs.
Totally agree.