I also just had a try-on appointment and am sticking with the 38mm.
It is true that the 38mm looked smaller than I expected. Both watches were also chunkier/thicker than I expected. On my wrist (165mm; width is the width of a credit card, whatever that is), the 42mm felt like a behemothI felt like it would truly get in the way of things during the day. The 38mm just felt more proportional, which I think is what a lot of folks who say "the 42mm looked just fine" are missing. "Fine" might mean that the 42mm is not spilling over the edges of your wrist, but that doesn't mean that it looks proportional or in any way sensible.
Here are pics of the milanese on my wrist. Top is 38mm, bottom is 42mm.
Image
Image