I restored my phone ahead of a midnight UK purchase, had no apps open other than Infinity Blade 2, turned on Airplane mode to cut out all incoming signal to the phone.
First two fights, yeah they went ok ... first fight in the tower. Slideshow. And that's regardless of the Rebirth level, regardless of the enemy being fought, regardless of their size, regardless of whether you reboot the device or not.
It's abysmal, and wait until you fight the giant Mech on one of the paths, yet more lovely slowdown. Check out Epic's forums, it's rife.