I don't see why it couldn't be done, but there are a few issues to deal with.
Is the card reader slot on the back of the Mini USB2 or USB3? When I check System Profiler on my own late 2012 Mini, clicking "Card Reader" doesn't give much in the way of information.
Having said that, there are now relatively high-speed SDXC cards, and USB3 external card readers. Using such a combination might not be as fast as using an SSD mounted in either an external enclosure or USB3/SATA dock via the USB3 ports, but it would certainly be as fast as a USB2 drive, probably somewhat faster. In other words, "fast enough" to be practical for a backup or emergency situation.
There are also USB3 flash drives out there that now boast relatively fast read/write speeds, as well.
To the OP:
I wouldn't use an SDXC card or flash drive as a "regular" external boot drive. You'd be FAR better off to get an SSD, put it into an enclosure or dock, and let that serve as your "external booter". In fact, that's just what I do.
BUT -- it might be practical to use SDXC or a flash drive to serve as an "off-site" backup. The idea is to keep a copy of your important stuff -away from- the location where the computer itself is. This protects against things like house fires, theft, etc.
I'm moving in this direction myself. Trying to decide in the next couple of days which might be the better choice for such a backup:
1. SDXC card and external USB3 card reader
or
2. USB3 flash drive.