We also had iPhones and various Apple computers which made us think hard about the iPad. Now we have both an iPad 3 and an Mini. Of all of them, I think the Mini is the most useful most of the time. First of all, I am in the camp that says there is no practical difference between the image quality on the mini and the image quality on the retina iPad. This is not a universally-held position, so I'd counsel you to look at a display Mini at a store. I'd suggest that you just immediately go to the Mini and see what you think of the screen, and only then compare it side-by-side with an iPad 4. You may find, as I did, that while you can tell the difference when the two are juxtaposed, when you are looking at the Mini there are no issues at all.
The size of the Mini is better than the full-sized iPad for transport and for holding for long stretches, and it seems a lot lighter than the specs would lead you to expect. Also, once you're using the Mini to do anything, you forget about the size of the screen. The iPhone screen feels cramped, but the Mini's does not.
If you'd use an iPad to read books the Mini is fantastic; easy on the eyes, and never tiring to hold. The full iPad is fine too, but the extra screen space hardly matters, and the weight does get annoying sometimes.
Unfortunately, the Apple Store experience is not the same as at home. If you're leaning toward it, though, take one home with you and see if and how much you use it. If you can't bear to part with with it, you've made your decision, and if you're indifferent you can still return it. At the price it's hard to justify owning one you'd hardly miss.
And as for asking advice on here, I think you're wise to gather the most information and opinion you can. Even if you were only considering an iPad it's helpful to hear what others have experienced, and I think this thread gives a good representation of opinions across the spectrum.
