You’re not wrong, but it does depend on the exercise.
I avoid presses as I expect that movement will smash anything on your wrist, but things like rows, swings or Russian twists use a different grip and I have got away with it so far.
TBH I expect my watch to be at risk of damage every time I work out...fact of life
Elevate your Apple Watch workout experience with ActionSleeve, a versatile fabric armband that allows you to wear your Apple Watch wherever it's most comfortable.
I've had a kettlebell hit my watch a few times doing KB snatches or something. Luckily it didn't do any damage but it scares the crap out of me. I always turn the watch so it's on the bottom of my wrist and out of the way.