Tap on Settings, General, Personal Grooming. Slide the setting for “Tweezers” from Auto to Manual and you should be fine.
Seriously though... didn’t know this was a thing, but the article that rugmankc posted shows you are not alone. Does it happen more for extended calls? I think it’s more comfortable to talk using the earbuds, but that’s not always convenient for short conversations.
I agree that a case is probably the way to go. Most are seamless one-piece designs.
This makes me concerned about the waterproof/resistance of the iPhone X. Yes there is likely a seal under there, but if a gap is wide enough to catch hair, seems like water could get in as well. I suppose that’s not a issue if the seal is below it though.