Took it back, they had a look at it and clearly something was wrong with it. Exchanged for a new one. After spending some time with the new one: IT DOES IT TOO! So annoying and a colossal waste of time. So either I get used to it, or take it back and figure something else out. Not impressed!
I backed up, DFU restored as new, then tested it and it's still doing it. So my guess is they all do it, or I just have the worst luck. I guess I'll see if I can live with it, or return it and get an altogether different phone.