System Preferences - Accounts - Login Options - Enable fast user switching
Then up in the top right you should see a little icon of a person or your username depending on the options, click that and you will be able to log into the other account while the apps stay open in your original account.
Since the applications of other users might be running in the background, does fast-switching slow down the computer? How about its effect on memory usage?
not really. depends if your using memory & CPU intensive apps in the other account. you can view the activity for each user by opening Activity Monitor in the Utilities folder.