Maybe I'm being paranoid here - but I'm assuming you still need to 'prep' the phone (double-tap the home button etc..), merely being very close to an NFC tag isn't enough for the payment to go through?
(I don't have an iPhone X so I've no idea how the authorisation happens there.. Do you just look at the phone? If you get some kind of notification that an NFC tag is 'seeking' payment, and look at your phone to see what it is, does that authorise the payment?)