It's been cultivated by using high-quality materials and caring about "fit and finish" (including on the software side) which is what any upmarket company should try to do.
The way the original iMac looked and felt (even though it was plastic!). The way the old Mac Pro tower looked and felt (I can feel the aluminum now!) not too mention how silent it was. The solid metal back on the original iPod. The aluminum on the nano. The curves on the redesigned nano a few years later. The way curves of the original iPhone felt. The way the screen was attached. The responsiveness of the software. The pure simplicity of the new Mac Pro.
SO many things go into making something "up market"... certainly capabilities is a piece of the puzzle but not the whole thing.
Let's go for a car analogy.
If you take a Ford Fusion and get the big engine option and give it a $60,000 price tag... does that make it more of a "luxury" than a base level BMW or Audi that has a smaller engine and costs less? No.
Price is not the only defining feature of "luxury". You can put a high price tag on a turd... that doesn't make it luxurious... nor does it make people want to buy it. You gain "cult" following for your "up market" items by proving yourself over and over again in your _design_ (where design is aesthetics coupled to capability... both have to work together).
The apple watch will be so much closer to a "luxury" than anything else on the market. It will have the "fit and finish" that comes with buying an Apple product. That includes the materials... the physical way you interact with it (like the unique clasps)... and the way the software works. All of that will be better... and I'll be wearing one

----------
BTW: These things _are_ competitors.... because successor or failure in the wearables market could cause people to choose one ecosystem over the other (or even switch!).
If you're just thinking of getting into the smartphone game - or you've been in one ecosystem and it's time for a new phone and you're interested in a smart watch... then a good smart watch to go along with the eco system you're going to buy into could be a differentiator.
Saying that this isn't a competitor is like saying the iPhone6 is not a competitor to the Android phablets because it runs iOS instead of Android. I think the "switcher" number from the 6 and 6+ are showing that people _do_ move between the ecosystems...