It is a worry if a malicious user stops your access to the phone. Lot of people are there who use the phone for their day to day office stuff. Not everyone buys a phone to send silly tweets.
That wouldn't be possible even with the full FaceID data. FaceID happens locally, not remotely. A malicious user would need to hack your phone at the hardware level to use this.