Is this some US thing where you need a beard and drive a truck to be considered a man? Never heard the "color X is for women" argument in Europe.
Traditionally, female babies in the US are sent home from the hospital in pink and boys in powder blue.
This would be easily noticeable in a baby and child clothing store.
Also, if you walk through any toy aisle you can see specific colors marketed towards boys and girls. Dolls don’t typically come in navy blue boxes and action figures in pink ones. This certainly carries through adulthood in many instances.
Gender fluidity is ever changing (and has been). In Romantic (primarily European) languages some colors have been innately masculine or feminine for god knows how long.
Ironically, this whole topic is a “grey” area.
I think your beard and truck example is a misconception, but I can understand why you might think that from outside. I have pinks, purples in my closet amongst a lot of other colors.