I had the problem, but it has been non-existent since the 4.1 update. I would suggest making an appointment with a genius and seeing if you can exchange the phone, especially if the problem is that bad.
Though, they'll probably asked if you tried restoring or resetting the settings. So if you haven't tried yet, I would try that first.