I know exactly what you mean. My second replacement iPhone 3G had the mushy volume button. Basically, it means that it has not been glued on properly to the rocker which presses on the microswitches underneath.
It is extremely annoying, mainly because there is little tactile feedback (no click) and it wobbles. I had to check the screen always to see if I was actually changing volume.
I also found the volume changing all by itself when it was in a case.
So, to the person who said "good god" this is a real issue!
To the OP I say ask for a replacement, especially on a new device.
Premium price means premium quality and anything less is basically unacceptable, that's the bottom line.