Exactly why I've never upgraded from my iPad 1.
I want all positives when upgrading, not any negatives. The weight of the iPad 3/4 and the charging times are worse than the iPad 1.
Improve those both, and I'm excited to finally upgrade my 1.
EDIT: Another poster wrote that the iPad 3/4 weight is actually LESS than the iPad 1. Odd. Doesn't feel that way when I hold them. Either way, it's heavier than the iPad 2, thus not everything is the 'best that you can get'. Charging time is still longer though.
In any design there are trade offs. For me retina screen beats any added thickness or weight. That's why I hope Apple moves ahead with a retina mini and also continues to sell the non retina mini. That way those who care most about weight and battery life can stay non retina and those who want the better screen and are willing to give up some battery life will also have an option.