Yes, you have to quit an application instead of just hitting the red button. If it's still on your dock then it means it's still running. This is useful if you want to listen to music using itunes, but you don't want the window open or minimized. Try it out, play a song then close the window.
Correction, it it still has the little blue indicator under it, then it's still running. You can place all the applications you use regularly on your dock, but if you open an applications you don't have there, it'll automatically be added at the end of the dock. It'll go away once you quit the application.