Restore probably won't help since it's a bug with iOS. Replacing the device would just be pointless and a waste of time since it's not related to hardware.
I used to have this happen fairly frequently with my 3GS but I don't believe I've seen it with the 4 yet. If it has happened, it can't have been more than once or twice 'cause I don't remember it at all.