Umm... if someone has the time to watch you enter your 4-digit passcode, which is a rare occurance if you have a Touch-ID/Apple Pay enabled phone, gain possession of your phone, go into settings, add their fingerprint, then make purchases at a vendor that supports NFC, wouldn't it be FAR easier for them to just take your card out your wallet?
If they have access to you and your phone like that, it would be a much easier job to just take your card.