I'm on the fence about this one.
I find my iPad 3 to be slow with iBooks (particularly ones with audio) and other apps (like DJ, iPhoto, SnapSeed, etc). However, I did not plan on purchasing another iPad this year.
My iPhone 5 with its A6 performs much better when using those apps (apps are more responsive, they launch faster, iBooks doesn't slow to a crawl, etc).
It's a tough call for a serious iPad user that would actually see a benefit from the A6X processor, but I feel like they're sticking it to me.
Then again, my iPad is now my primary computing device.