its likely the cover-to-mute feature. my wife has this issue too. when she crosses her arms, her right forearm actually covers the watch on her left wrist. this trigger mute mode and it doesnt come out of it unless you manually do it.
she's decided that mute mode is better anyways (she doesn't need the sounds from the watch).
for me, i have a similar issue in that when i cross my arms, sometimes it trigger the weather complication i have on the top left corner. so i often find my watch on the weather app, instead of the clock (and i have "wake to app" set instead of "wake to clock").
