You can keep changing the battery as I do it, the issue you encounter is the battery will default to 01.01.1970 so you'll be time travelling and unable to connect to apps like twitter as certifificates will become invalid by date.
You can correct this by setting time manually to right time until you plug into Itunes again.