Wipe the dirt off the holes with a dry cotton swab or toothbrush. Spray some rubbing alcohol onto the cotton swab and then wipe off the dirt from the holes. Use a damp cloth to remove excess water from your watch band.
Every few weeks I grab my straps and pop them in an ultrasonic cleaner with a bit of cleaning fluid, sometimes the fabric ones need a toothbrush scrub but apart from that they come up great.