If there is audio, location (think gps), or voip (skype) its running in the background. Otherwise, it is saved to memory in the current state, if the app supports the fast app switching api. They can also do task completion in the background and local push, though those don't go in the background for long.
Edit: I forgot to add that just b/c the app is in the tray at the bottom doesn't mean its running, thats just the UI for multitasking.