I had the same problem and did an exchange. Now experiencing the same issue with the replacement. What I do now, which seems to resolve the issue, is knock the button with my knuckle. Sounds lame but fixes it.
The Apple guys thought I was on crack when explaining and demoing the problem. I figured it was OCD. It's to know I am not the only one experiencing this issue.
It's like when you press it slightly you can hear first click but nothing happens just the ring rubbing against the glass edge then you press harder and you hear second click that makes the connection.