But if I switch out of the chat client to another app, it will disconnect me right? So how does that give me the ability to stay online? It doesn't.
It all works fine. When you connect with Beejive, or Meebo, or AIM, your phone is connected directly, and whenever it switches, Apple becomes a connection broker for those existing connections. Once a notification is sent from AIM for example intended for you, if the app isn't up, it hits Apple requesting that the message get passed off to the push notification service on the iphone, and thats when it prompts you. When the application comes back up, it retrieves the message, displaying it for you.
It all works seemlessly. Beejive, and Meebo in my experience have done the best job of this so far in making it as uncomplicated, and flexible as possible.
Long story short, Apple keeps a persistent connection to your phone, that provides services to ping your phone, and prompt you for response when there is something ready for you. This is beneficial, because instead of having a background application for everything constantly running for quite literally everything from Tap Tap Revenge, to We Rule, to BeeJive, and AP, and so on and so forth, only foreground applications, and Apple Push Notification is are the only ones constantly running.
The new things introduced in the new iOS4 extends this well beyond just push notifications, and has things like state saves, that will freeze your state on exiting an application, background audio service, background navigation service, local notifications, and task completion.
Whats really interesting is how Apple juggles all of the tasks in memory. I'd recommend to anyone that wants to get their head around this to check out the WWDC videos that are now up.