I suggest those that think their SD slot is fine, test it more throughly with all your SD cards.
I'm the exact opposite of an expert on this, but I believe this is exactly the crux of the compatibility problem.
I don't yet have the 2021 MPB but I've had problems reading some SD cards with my 2015 MBP that read just fine with a USB connected SD reader. The issue is that the SD "standard" is anything but. It's something that grew out of previous efforts.
See this Wiki page https://en.wikipedia.org/wiki/MultiMediaCard for an older version of SD, just slightly mechanically different. If SD slots attempt to accommodate all these subtly different cards they can have slight mechanical differences that keep some cards from working.
Here is a picture of just one of the slight differences: https://en.wikipedia.org/wiki/File:MMC_card_vs_SDHC_card_undersides_angle.jpg