Alright, for anyone interested to find out how this turned out: I tested the XR and the XS side-by-side for three days. Almost from the first minute, I knew XS was the way to go for me. As I said in the beginning, really my only concern about the XR was the size. The XS is not that smaller or lighter if you just compare the specs, but in person, as it were, it is a huge difference. The XS is a large smart phone, the XR is a phablet. It really felt like a brick in comparison, especially if you add a case. Most importantly, I can use the XS almost entirely one-handed.
While testing, I have also come to like a couple of other things about the XS which the XR lacks. First, the display is indeed sharper and the blacks are blacker, which is nice. But really, I very much liked the LCD on the XR so this would never have been a dealbreaker. Also, the whites on the XS are indeed somewhat yellowish, which is manageable, but certainly not a plus.
Second, I am quite happy to have the dual lens camera system. Portrait mode with the XS is a whole other story than with the XR. It is not only that you can take "portraits" of everything, not only persons (which is especially nice to have if you want to take a picture of a person who happens to look in the other direction). It is also the simple fact that you use a portrait lens for a portrait and not a wide-angle-lens, which is something a photographer would never dream of. Portrait shots thus turn out really nice and undistorted. It is true the XR lens works better in low light, but this advantage pales in comparison to the disadvantages.
Third, and most unexpected, I found the XS to sound much better than the XR. The speakers on both devices are great and (almost equally) loud, but the XS produces a much more balanced and nuanced sound. The XR is somewhat heavy on the trebles, while the XS sounds natural and you can actually hear much more detail in the music. I am bit of an audiophile, so not everyone might hear this or share this impression, but I am actually quite surprised that this has not been much of a point of discussion before. Of course, there are much better ways to listen to music than through your iPhone, but sometimes it's your only option and it is nice to have a great sounding device for such occasions.
In sum, I am very happy XS user now, enjoying even more about it than the size alone. However, none of the additional benefits would have made me get the XS over the XR. The XR, to me, remains a wonderful option for anyone who is happy with the size, which is a very individual thing indeed.
congrats on figuring out what worked for you. i came late to this thread but was curious to see what you ended up with. it sounds like arrived a happy place, also the same place i convinced myself to be after the keynote address in September.
i have owned the 6 and 7, and handled several friend's X phones in the past year. i knew the X was the max size for me even though i have large hands. i can just barely access everything one handed on the X (and Xs) so i went with that even though my subconscious kept balking at the extra $250 i paid to get the slightly smaller form factor. but i knew by specs alone that the XR would force me to start doing things two handed which i didn't want.
one thing i have noticed since owning the XS (even though i had handled the X many times before) is that it is indeed much denser than the 6 and 7 i've owned. the footprint of the XS is just slightly more than the 6/7, but it weighs considerably more. it took me a few weeks to get used to it, but i eventually did. (it is noticeably heavier in my pocket which to me was a pretty big adjustment)
like you, the OLED and second lens are nice bonuses. i like them, but if it weren't for the size factor, i would have certainly been willing to buy a small XR and save myself $250. for my wife's sake, i hope Apple will at some point release a smaller XR to replace the SE. she inherited my 7 and it was definitely a fair bit of adjustment due to the weight and size differences between the two. i had originally thought i'd gether with an XR for Christmas. however, i won't -- even though she uses the 7 two handed for a lot of things (she has small hands), she says she can't imagine wanting to handle a larger/heavier phone.
(when i pick up the 7 now, it feels like a feather!)