Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
I'm about to just hold onto the phone and wait for them to contact me if they want it back.

This.

What I'd do is send formal communication to them (shoot an e-mail to all customer representatives that you've been talking to) where you'll:

1) Explain the situation;
2) Explicitly note that you are willing to return the device and have acted in good faith to do so;
3) Note that you still haven't been able to do so because of Apple's incompetence;
4) Request formal and written instructions to return the device.

I think you should be covered. Honestly, I don't think Apple will pursue this too much and in the end I think they'll let you keep it - it'll cost them more than the price of one phone to handle such an exception in their system and allow that phone back in stock (think of how many supervisors would need to clear that). So keep it for a while, maybe you've got a new spare iPhone you can take to Coachella or Tomorrowland or UMF or something.
 
1 - Put it in the drawer and just wait for them to request it.

2 - Keep all your WRITTEN documentation, and good notes on any discussions.

3 - See #1
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.