This already exists: The Digital Camera Connection Kit includes adaptors for reading/writing to SD cards and USB devices. And it's $30! (Only for 30-pin dock connectors, it looks like. No Lightning version yet.)
Apple would only need to implement OS support for file systems other than those in DCIM format. And support for iPhones too, not just the iPad. Not too likely, but one can dream...
The Camera Connection Kit's SD adapter is a read-only device. There is a Lightning to SD Card Camera Reader, just not the complete Camera Connection Kit with USB adapter.
As far as I'm concerned, all Apple really needs to do is add a write-enabled SD card adapter as a "share" option (any file that can be shared to email, Twitter, etc. should be OK to share to an SD R/W). At least, that's all I think they'd ever be likely to do. They're alway going to be hinky about something that could turn iOS devices into music file-sharing machines (either import or export).
I think they introduced the camera connection kit on the reasonable premise that photographers would like to edit and display their work on the big iPad screen, and promoting such use would make the device more desirable. Since the original iPad had no camera, it needed to speak to a camera in a manner more elegant than, "Sync camera to computer, sync iPad to computer" (I know that's why I got my connection kit way back when).
Why isn't this supported for iPhone/iPod Touch? You could just as easily ask, "Why isn't the Stocks app included in iOS for iPad?" Apple has its vision for how it expects (and wants) each of its devices to be used. But, with iPhoto available for the smaller screens, they don't exactly have a consistent case for "iPhone owners don't need that."