Just read about a really good safeguard against this using Screen Time, actually. I'm having trouble remembering where I saw it, but the basic idea was that you can go into Screen Time > Content & Privacy Restrictions and then disallow Passcode and Account changes. Once you've done that, you can ONLY change your Apple ID settings if you know the Screen Time PIN, which is a 4-digit PIN you can make totally different from the PIN you unlock your phone with.
I just kind of battle tested my own phone to see what I could do with a piece of tape over the FaceID sensor, just using the unlock PIN. I can get into the phone, but my banking and credit card apps all need FaceID or their own unique password to get in. Same with Venmo. I'm not using iCloud Keychain, so there's no danger in anyone using my unlock PIN to get at those banking passwords -- they're saved in 1Password which requires either FaceID or its own (very long) password. The one remaining way for people actually send money off my phone without FaceID is Apple Cash, which does not have a PIN option and which I don't believe can be protected with the Screen Time PIN from what I can tell. Kind of lame, but I'm not going to disable Wallet entirely.
It would suck to lose money, but honestly the HUGE thing here is not losing my entire Apple ID if someone PIN jacks my phone. I still feel like this is a fairly unlikely thing, but it feels good to have at least a layer of protection there.