Ideally you would want to test your apps on all iPads available.
At the bare minimum you want to test on the oldest supported device.
What we like to do is make sure it runs on an iPad 2, and an iPad3. If your app runs smoothly on those, then it should be fine on the newer ones.
Note, that there are certain features only newer iPads support. For example i iOS8, apple allows users to use the frosted glass effects in their apps. This only works on newer iPads (i think the oldest they work with is the mini).