I believe the ATT LTE SIMM (that comes with your iPad3) is required for LTE service. You can swap SIMMs with your iPad2, but you will only get 3G service. You iPad will say "4G" but will still be 3G. Only if the iPad says "LTE" do you have LTE service.
Apple/ATT make it easy to transfer your iPad data service. Just fire up the iPad3, restore your old iPad data using iTunes, then open a browser and it will prompt you to xfer your data service. The data plans will say "LTE".
After I xfered my data plan to the iPad3, my iPad said "LTE" and a speedtest shows 10-18 Mbps up and down with only 2 bars of signal strength. Nice!