It's probably more along the lines of the watches didn't catch on as a fashion item by people who buy fashion brands like they had hoped. The watches are doing ok, but still as a tech item not with the Vogue crowd. So now there probably wasn't much of a place for him if things weren't going in that direction.
Just because you don't see it, doesn't mean that it hasn't caught on with the fashion crowd. I'm a photographer, and I do frequently see Apple Watch in the fashion industry as a "show off" item that influencers wear in a
future fashion trend where technology and fashion are merging. Fashionistas already make a point of showing off their iPhones as part of their look, often with expensive phone cases. The Watch is a more natural extension of that. It's exactly what Apple had in mind when they hired major fashion brand CEOs to take on important roles in the company.
A little story: I was shooting a luxury watch media day where the CEO of Chopard was demonstrating his company's watch lineup to fashion writers, one-on-one. Before the first reporter arrived, him and I had a conversation about Apple Watch after he noticed it on my wrist. He said that he didn't think it was threatening the traditional watch market. "Not many people buy it", he said. I agreed with him on the point that watch collectors and luxury watch lovers wouldn't stop buying mechanical watches but I told him that the iPhone wasn't an overnight success either despite the hype. It took 3 generations before iPhone started becoming as ubiquitous as they are today. The first AppleWatch sold far more than the original iPhone did.
The first fashion writer arrived. From Elle magazine. As the CEO went to put a Chopard watch on her wrist, she raised her sleeve and there it was, a rose gold Apple Watch with a pink strap. Him and I looked at each other and I nodded. Next, a pair of reporters from Toronto Life arrived. Between the two of them, they were wearing 3 Apple Watches. The man was wearing an Apple Watch on each wrist! I had a broad grin on my face and the CEO knew why.
They kept coming and just one writer came wearing a Chopard watch already on her wrist. When the last one left, the CEO seemed to have changed his mind. I suggested Chopard get into the luxury smart watch strap business. I'll really laugh if Tim Cook introduces him at an upcoming Apple Watch event.