I think this is a definite case where "it ain't what you do, it's the way that you do it". If he glances over as an attractive woman walks by, it's harmless - he probably couldn't stop if he tried! If, on the other hand, he cranes his neck around to get a good look at her bum as she walks by, then fair enough, you have reason to be annoyed.
I have a few female friends who've told me if they're walking hand-in-hand, they can always tell when their boyfriend spots someone attractive as they'll squeeze their hand a little as she walks by. You can just imagine the near-subconscious thought process: "Oooh! She's cute... woops, must not annoy girlfriend, give a reassurance squeeze.."
I have a few female friends who've told me if they're walking hand-in-hand, they can always tell when their boyfriend spots someone attractive as they'll squeeze their hand a little as she walks by. You can just imagine the near-subconscious thought process: "Oooh! She's cute... woops, must not annoy girlfriend, give a reassurance squeeze.."