If it's getting sluggish (and wasn't before), then clear you cache, quit Safari, and remove the contents of ~/Library/Safari/Icon/ as devilot said. That will help speed up Safari considerably.
Also, if you've updated the OS recently it may have included Safari updates and if you use Safari plug-ins that are out of date, that could also contribute to Safari being slow.