I really feel Apple should have put a bigger battery in the SE, 2500mAh minimum!
Sorry if I have asked this before, but do you have Background App Refresh turned off? Also is Siri busy listening in the background? I think those features could cost battery life on the SE.
It did a number on my SE2020, which already has a shorter battery life. I could actually watch the battery drop by 1-2% just sitting there. I did the 14.1 update, and all is well.