I've had a Pebble smart watch for a year and a half now and I only keep my iPhone on silent (no vibrate) when I'm in a meeting or in class. Otherwise I keep the sounds on.
With the Watch I don't know yet - I'll be interested to see how the notification sounds actually sound coming from the watch and if they're bearable I'll probably just leave my phone silenced. Too early to tell now though.
I haven't found my pebble to be reliable enough to do this. For some reason, the original and the Steel, just decide to unpair themselves whenever they feel like it! I hope watch is a little more reliable!