Fragmentation hasn't hurt Windows. And it hasn't Android.
I would say fragmentation is the "Internet Explorer 6" of Android.
Internet Explorer 6 doesn't support many features that are necessary for modern websites, but it just refused to die and make room for better browsers.
As it is the standard browser in Windows XP and many users don't care about upgrading or can't upgrade (PCs in workplaces), this lead to fragmentation of browser features and many web developers had to do extra work to get their sites running on IE6.
This kind of fragmentation is now also visible in Android, with more than a third of devices not ready to use the many improvements that Android 4.0 has for users AND developers of apps.
And as one of the most important apps - the browser - is not compatible with versions below Android 4.0, we have the IE6-situation again.
Google started development of Google Chrome in the past to make sure that everybody has access to a browser that supports all the newest web technologies, so people would be able to use all the new Google web services.
And now the users of a Google product are unable to have the best web surfing experience (and compatibility) possible.
As i already mentioned, there are more people running iOS who are able to use Google Now now than people running Android 4.1 and higher who would be able to use it.
Android feels like a test environment for interesting new features, that only few people will actually be able to use until a year after the Android version containing this feature has been released.
When Google wants a large number of users, they need to go "iPhone first", because that's where the highest number of possible users is, not Android.
When your competitors platform is the one that earns you the most money (ad impressions for Google, Microsoft making more on patent licensing fees for Android than with Windows Phone 7 - and probably also WP8), you have a problem.