Display size and keyboard size. People with big hands could never hope to type on the keyboards optimized for the iPad Mini. There is also the concern with the screen size. Some people cannot see small text as well and others who have eye issues in the first place need larger displays so they don't strain their eyes.
I do agree that the iPad Mini is more useful and convenient than the full-sized iPad. I use mine at work every day. But, I do know why some people don't want one or shouldn't use one, as opposed to a full-sized iPad.