I don't think you understand just what you're looking at.
The electrical contacts on the port are self-wiping (that's the term used in the electronics industry - it means they clean themselves each time you plug-in and unplug), and are on the upper and lower surfaces of the port, not at the far end/bottom. It's very unlikely you're able to see dust/dirt on those contacts, due to the viewing angle and narrowness of the slot. Because of their design, it's very unlikely they have dust/dirt on them.
What you're probably seeing is meaningless dust/lint on the far end of the port - as long as the connector still clicks into place, you don't have to worry about that. Eventually, pocket lint may build up at the bottom of the port and maybe it will keep the connector from clicking into place. To clean out that lint, I recommend using a thin, flexible, plastic pick (like the tooth pick/flosser I've shown below. Be very gentle - just enough to dislodge the lint from the bottom of the port.
Do not use the SIM tool - it's too stiff, and not well designed for the task.
View attachment 634289