Totally agree on the aesthetics, this is really pushing into the "fashion" area of personal products. I could see a really beautiful design with a stainless outer band, maybe white or black options for the face (like current devices), but people _love_ to customize, so if I were Apple, I'd definitely make sure the design was "friendly" to cases/covers.
Build in mount points, or even make the product out of the box be modular ... heck, I could even see it coming in a box, on one side is the core electronics/face, the other side the bezel/case, band in the front. Apple offers colored (anodized) aluminum bezels/cases, and it opens up a whole 3rd party market.
Totally with you exception being the case. I don't want an old school Swatch with interchangeable faces, I want a nice watch. I won't be putting various silicon faces on the watch.
I do wonder if any watch that post-dates Apple's creation will fall victim to Apple's love for testing patent law (though patent law sounds like an oxymoron to me now).