Storage space and, at some times, CPU speed.
The first generation iPhone originally had a 620 MHz CPU underclocked to 400 MHz; the first generation iPod touch also originally had the same CPU with the same underclocking. Firmware version 1.1.2 increased them to 412 MHz.
The 2nd generation iPhone (iPhone 3G) used the same class of CPU and the same degree of underclocking as the updated firmware on the original iPhone -- 412 MHz; the 2nd generation iPod touch, released some months later, also used the same CPU but at a higher clock speed -- 532 MHz -- thus placing the iPod touch at an advantage over the like-generation iPhone. This is an anomaly in the history of comparative specs of the iPhone and iPod touch.
In both the 1st and 2nd generation iPhone and iPod touch, they always had essentially the same GPU capability.
With the respective releases of the 3rd generation iPhone (iPhone 3GS) and iPod touch, the two systems had identical CPU and GPU capabilities.
It is reasonable to assume the same will be true of the 4th generation iPod touch when it arrives -- having equivalent CPU capabilities as the iPhone 4.