I also have the same issue happening to me. My phone is about 3 feet away, but super close to my watch and when I say "Hey siri" both of them respond so OP you are not alone.
In a
live interview on WWDC 2017 Craig talked about making significant improvements on this issue, but it seems more improvements on this issue are necessary judging that many people are still affected by this.
The only solution I can really think of at the moment without having to turn "Hey Siri" off iPhone is to press and hold the digital crown when you want to use Siri on the watch. It's not ideal as it takes away the convenience of the hands free experience, but until solves this problem entirely, it's the only solution we have without significant trade offs.