To my understanding, smaller shops make a very small profit margin on new CDs, which is why some smaller shops often do things such as: used CDs, used and new movies, Laserdiscs, books/magazines, food, Zippos, vitamins, energy drinks, smoking accessories, new and used vinyl, Beta, limited edition/valuable editions not seen most places, electronics, etc.
Some of the oddball stuff they carry they seem to get from a variety of small retailers or publishing companies that makes or remakes niche market/cult following media. Some also purchase from individual collectors for out of print stuff.