From the manual...
When set to silent mode, iPhone doesnt ring or sound any alerts or sound effects (alarms set using Clock do sound). By default, when you get a call, iPhone vibrates whether its set to ring or silent mode.
If iPhone is set to ring mode, you can silence a call by pressing the Sleep/Wake button or one of the volume buttons once. Press a second time to send the call to voicemail.