Yeah, I had the same problem. Easily fixed.
Just to expand in case you need more help, you have to have your iPod enabled as a harddrive, as you would have done before. So you're iPod appears as a Harddrive on your desktop.
Double click on the iPod icon on your desktop, then open the contacts folder and delete all the files in this folder.
Then iSync your iPod again, and problem should be solved.