There's no way around needing to just try both sizes on in the store. I'm a guy with medium-sized wrists. When I bought my Series 0 way back when, I went with 42mm since I just assumed 38mm would be tiny and was more intended for women (weird assumption, honestly). Within that year I stopped wearing the Apple Watch entirely, as it was too sluggish and I didn't like having a huge black rectangle covering my wrist (what everyone else sees, really). I moved on to some great traditional watches (hello Nomos!) and along the way got used to a smaller size.
About four months ago I picked up a Series 2 Nike watch as a running watch, and when trying both sizes at the store, the 42mm just felt too huge. I liked the sense of the 38mm being light and easygoing. Went with that, worked great for running. A few weeks ago I decided to wear it as my all-day watch again to see if I'd want to jump back in with a Series 3. Turns out, between watchOS 4 and the smaller 38mm size, I was FAR less annoyed with the watch on my wrist all day. Who would have known the 38mm size was what I really wanted all along; no more feeling of "huge screen" dominating my wrist. Now I'm sporting a 38mm Hermes Series 3 and enjoying it a bunch.
Anyway, point is, both sizes work great; it completely depends on what YOU like on YOUR wrist. Try both and trust your gut.