If you've pasted the icons manually then just select the icon, press command-i to get up the information window, click on the mini icon in the top left hand corner of the 'general' section, and press command-x to remove it.
If you've changed the icons using the Iconfactory's CandyBar app, there's a 'restore default' button at the bottom of the app window that will do the trick.
Hope that helps.