They cannot pinpoint YOU because data is sent anonymously. They can roughly pinpoint A phone, but don't know whose phone it is because the data is sent anonymously (aka without identifying information)
I think it's not as bad as what the media would have you believe, BUT it is worse than what Apple wants you to think.
Sure, cell towers could be up to 100 miles away. And when I ran the mapping tool and plotted my locations, and zoom in far enough, I do indeed see a grid of cell towers as opposed to actual locations where I've been standing. All anyone could know is that I've been "somewhere" in the vicinity.
(And this isn't new. Some time ago I came upon a car crash and called 911 on my cell phone to report it. They were able to get the location to send emergency services just by where I was calling from. It wasn't 100% accurate -- they asked if I was near a major intersection and I told them it was about a block from there.)
However, if it's also tracking wifi hotspots, those can pinpoint you pretty closely. Most people stay within 30-50 feet of their wireless router, and the ones you spend the most time connected to will be the ones at home, at work, and and at your friends' houses.