I have an iPad 3, so I know it's not the same as the iPad Mini, but I have not used a case for my iPad when it was inside. I just have a neoprene case for transporting it. I don't really use it outside of the house, so I hardly use the case at all.
Personally, I think you would be fine - as long as you don't drop it. If you do drop it on something hard, be prepared to have damage ranging...