Likely: think of this like switching from AppleTV to a game console or DVD player: Google TV apps are their own source. So when you want to switch back to AppleTV, you likely need to switch back to the AppleTV source (the HDMI input into which AppleTV is connected). If you had 2 physical sources- AppleTV and a DVD player in HDMI1 and HDMI2 respectively and switched to DVD on HDMI2, to get back to AppleTV on HDMI1 you would need to switch the source/input back to HDMI1. There is probably an "input" or similar button on the TV remote to change inputs... or perhaps a cog wheel or similar button that can then get you to a list of choices of which one is "input/source".
The built-in GoogleTV apps are probably their own source, as it would be with perhaps a TV antenna or cable input and other devices hooked into other HDMI or not HDMI video input jacks. As your stuff is set up right now, you likely need to manually select the right "input" into which AppleTV is plugged if you switch to some other source of video. Note that if you are doing this and the screen is black, check the front of the AppleTV to make sure it is turned on (little white LED is on. AppleTV will often turn itself off after a short period of inactivity).
A remedy to manually switching is to get CEC working on both TV and AppleTV (and any other physical sources you have attached). Then, whichever device's remote gets a click should gain control of the TV screen. With CEC "ON" for everything, you could then switch to the Google TV apps or any other hardware source plugged into a jack and then just click a button on the AppleTV remote to make AppleTV "tell" the TV (via CEC) that you are now wanting to use AppleTV on the screen. My guess is that you don't have CEC turned on in either AppleTV, the TV or other devices.
If you don't want to use CEC, you'll need to manually change inputs when you want to switch from one source of video to another.