FaceID is contained within the phone’s Secure Enclave, just like TouchID. If you’re comfortable using TouchID, there’s no reason to avoid FaceID.
Or to put it another way, all of your security concerns over FaceID exist with TouchID as well, so if you’re avoiding FaceID you should also be avoiding TouchID.
Or to put it another way, all of your security concerns over FaceID exist with TouchID as well, so if you’re avoiding FaceID you should also be avoiding TouchID.