This thread keeps appearing on the first page, just WOW!
This will blow your collective mind...the few that DON'T rattle are defective! There is something stuck under the power button and / or the case is not assembled properly. The phones manufactured to correct tolerances have buttons that move freely in their respective case openings.
I have a 4 and a 4s. They are the same way but the heavier stainless steel dulls the sound. The aluminum sounds more hollow and resonates a bit more.
The power button (and to some extent volume buttons as well) does rattle, but the screen and sound quality and more importantly the overall finish are good. Given that there are so many which rattle, doesn't seem like its worth taking a chance replacing it and ending up with some other defect and keep on replacing without enjoying it.