I am quite depressed about it. I've bought a few upgrades and returned them already because I did not like the phone for one reason or another, and now the AT&T store refuses to work with me. Each of these returns cost me $35 as well. I returned my 4S because it was crackling when I made calls so I was going to let my contract roll back and use an unlocked phone until the next iPhone iteration came out. Unfortunately that is not the case.
Honestly, you need to keep trying. I, too, returned a 4S to the Apple store recently and was assured by the retail associate that my upgrade would be reversed. Well, that wasn't the case.
I called AT&T and they blamed it on Apple. I called Apple and they blamed it on AT&T. Finally, after multiple phone calls and two weeks of time passing I returned to the Apple store where I purchased the phone. I asked for a manager and politely explained my situation. She called AT&T and stated that I returned the phone and gave them the IMEI number and Serial number. Then she said "all done." Like a boss.
Twenty minutes later I received a text from AT&T stating "upgrade reversal in progress." The following morning I dialed *639# and BAM! Upgrade reversed.
So, definitely return to the Apple store where you purchased the phone and bring your purchase receipt and return receipt. Be nice, remain calm and your upgrade will be reversed.