I can't say specifically if it has to do with Verizon getting the 4 or Sprint getting the 4S, but as an AT&T user in NYC (midtown, west village, soho, east village primarily) I will say service has definitely improved since I joined in 2009. It's still not great, but definitely better.
Speeds, especially on the 4S with HSPA+, are pretty good. I get 1-2Mbps in manhattan and 5-7Mbps down in queens. Definitely noticeably faster at loading pages, map data, facebook, etc than my 3GS.
In some buildings, and in midtown during the day, service may be very low, or it may say you have service but you'll be unable to place a call or send a message, and this can get very frustrating, but it doesn't happen enough to make me want to switch to a different carrier.
Overall I am fine with AT&T's service. Data speeds are generally impressive and definitely faster than Verizon and Sprint's 3G, and even keeps up with my friend's Droid Bionic on LTE loading data (a good testament of browser and hardware optimization and efficiency!) despite its higher capability of raw download speed.
If I ever have a problem, I make sure to send a report (or two) with the Mark the Spot app. Hopefully the more complaints they get about spotty coverage, the more likely they will be to work on improving our service here.