I had this problem too with duplicate "On My Mac" sections in Mail. Found the following fix posted by Wes Adams on the Apple discussion forum:
The easiest way to get rid of the duplicate is to just move "On My Mac" up in the list. Just grab it and move it to, lets say, above the "Trash" folder. The duplicate will vanish, then you can move it back if you want. The duplicate will still be gone. I don't know why, but this works.
http://discussions.apple.com/thread.jspa?threadID=1202501&tstart=45