X owner here. The Xs has a faster chip, better camera, better/faster FaceID, and a new color option. There are a lot of little things that will be important to some people but I probably won't notice in my everyday usage.
For me, these upgrades aren't important enough to me to justify the expense -- and it is a really expensive phone.
I bought my X through the Apple iPhone upgrade program. If I keep it one more year, it'll be paid off and I can sell it for $400 or so, and put that toward the XI or whatever they'll call the 2019 model.