Applications mostly are packed into so called disk images (.dmg) files, but they also come packed into .zip files, and Open "safe files" after downloading is selected, then .dmg files get mounted (icon in Sidebar and icon on Desktop, if you set the Finder Preferences to do so) and .zip files get extracted to the folder you specified in Safari (or your other browser) to save the downloaded files to.
Safari Preferences
Finder Preferences
After the .dmg file is mounted, you drag the application from that mounted volume into your Applications folder and eject (unmount) the disk image again and maybe even delete the downloaded .dmg file.
Then you start the application from the Applications folder.
If the disk image contains an installer, just run that, and after the installation, eject the disk image and delete the .dmg file if you want.
With .zip files and the extracted folders it's kinda the same way. Drag the application to the Applications folder or run the installer, and after that, delete (or not) the .zip file and the extracted folder.
The Dock is only a shortcut to applications or folders and files, so anything you drag on it, will stay where it was dragged from.
Also have a look at the following links, as the information presented there might be helpful in your future endeavours into Mac OS X and could clear up initial confusion and may even prevent harm to your system or your files.
Mac OS X Basics
Switch 101 - guide with articles made by Apple on how to accustom yourself, after you switched to Mac OS X from Windows
Mac 101 - How to get started with Mac OS X
Find out how - tutorial videos made by Apple on how to do certain thing in Mac OS X
Pro tips - tips made available by Apple for easier ways of doing certain tasks
MRoogle - a very effective tool to search these fora using Google and made available by
edesignuk, introductory threads:
1,
2 and
3