Because the feedback they were getting on the 2013 Mac Pro was leading to a clear picture that the iMac Pro (which came out of the same failed product philosophy that gave us the 2013 Mac Pro) will be a market failure of such epic proportions, that it will make the 20th Anniversary Macintosh look like the iPhone.
Apple has been taking flak for the Mac Pro's current design for years - long before they decided to replace it with the iMac Pro. Plus nobody knew the iMac Pro was coming much before it's reveal at WWDC (there was a vague rumor about a new "Pro" iMac from the Foxconn Leaker, but nobody seemed to take him seriously). And as much as the detractors refuse to accept it, iMacs are currently in wide use in many markets that would benefit from a more powerful model like the iMac Pro. All that being said, I do believe they came to realize that the iMac Pro, while better than the current Mac Pro, would not be sufficient to meet the needs of their most-demanding customers and that is where this new Mac Pro is aimed at.
What could have happened?
I think what happened is that the proponents of the Macintosh finally got their voices heard by senior management. Of Apple's three hardware "tent poles", only the Mac is seeing general growth year on year within each respective quarter (iPhone is generally flat and iPad is generally declining) and senior management is finally noticing this. As such, they're now willing to invest in Mac to a higher degree than they have in the past.
I hope that this time, Apple has done a thorough market analysis and talked with the end users and show them the prototypes for end user feedback. I think they lacked that with the Mac Pro 2013.
We know from the "April Meeting" that Apple has been speaking with some of their largest customers on what they felt were the limitations and drawbacks from the Mac Pro and what they wanted in a new Mac Pro level machine. The iMac Pro is a more capable machine than the current Mac Pro, but it's not going to be as capable as the most powerful machines offered by OEMs like HP or Dell running Windows and Linux. And this is where I expect the Mac Pro will be aimed at.