Games like Fruit Ninja and Angry Birds run no different from the 1 and 2. I didn't have a chance to run Infinity Blade or Real Racing 2 HD, but I've read it is a much better experience on the iPad 2.
Garage Band is a very different experience. It's not just the time to load, but also the time to switch between the instruments and process the various tracks. If you are a heavy GB user I recommend the iPad 2.
Loading and scrolling through larger Powerpoints on Dropbox and Keynote is very different. The Powerpoints would freeze and crash on the iPad 1 but not on the iPad 2. Granted smaller docs may work well, I just don't have any smaller docs running on the iPad.
The decision tree is 2 fold: Camera yes or no (not that important to most users)? High powered apps yes or no?
For the casual user you definitely should buy the iPad 1, especially if you already have an iphone for Skype and FT.
However if you have any thoughts of 3D gaming or use processor/memory intensive apps then do yourself a favor and buy an iPad 2.