Living in Brooklyn, this is a complete and total dealbreaker for me. It's not that I can't get around NY; of course I can. The problem, like many others have said, is the desire to have the shortest possible route to your destination so you're not sitting at a bus stop or subway for 15-20 minutes.
If you're looking for the shortest possible route, then you shouldn't be using Google maps anyway. I've stopped using it completely. In 100% of my tests, HopStop has given me the faster and more direct route, with less walking and less transfers. I've posted an example in this thread:
https://forums.macrumors.com/threads/1442987/
Once HopStop is updated to integrate into iOS 6 maps, we will have a much better solution than is currently available.