It sounds like you accidentally made an alias for Safari. Look in the Applications folder and make sure you see Safari there without the arrow. Then delete the one in the Dock with the arrow and drag the original Safari back into the Dock to put it there without the arrow.