The 3G can be restored/updated to any version you want so restoring will not update it to 4.0. The 3GS and iPhone 4 however can't be downgraded once Apple stops signing the firmware. Now 4.0 is out they stopped signing 3.2 so anyone restoring their 3GS it would update/restore to 4.0 because 3.2 can't be signed any longer. Does that make sense?
Basically because you have the 3G, if you restore your phone it shouldn't update the firmware at all.