compuwar has the perfect solution (as long as it doesn't destroy Aperture's stored metadata) Can't we develop this into a useful app? Could this be implemented in an AppleScript script?! Maybe an AppleScript could use compuwar's ideas, and send a "delete this photo" command to Aperture. That would at least make the manual checking of the slightly different photos a lot easier (i.e. it would kill all the definite duplicates, leaving just two or three - one with good metadata, two dupes - to manually work on)
Even better would be if the AppleScript could find the duplicates and near-dupes by efficient multi-pass: filename, file date, CRC head, MD5, SHA-1, etc. the data portion; then compare the metadata and pick the one with the most fields filled out, then mark those in Aperture as #1 Best Dupe #2 Less Quality Dupe (#3-18 deleted because they are exact dupes of #2)
Duplicate Annihilator is on the right track, and the Aperture version actually claims to use CRC and MD5, but the fact that it marks/deletes UNIQUE photos (it even admits so) is totally unacceptable. (has this been fixed recently?)
I have had to import about five versions of broken iPhoto libraries, so I now have up to 10 duplicates of many photos, but probably only one copy of many. My Aperture library now contains about 200,000 photos, probably 70% of which are dupes. I have put a lot of work into the meta-data that (I hope) got brought over from the iPhoto imports, so I don't relish losing that. At this point it seems that the only way I can clean this up is to devote about 150 man-hours to going through the entire library, clicking on each visually duplicate photo, finding the one with the good metadata, and deleting the other 7. I am utterly bemused that these "computers" that were supposed to save us this kind of horribly repetitive manual labor are unable to do this, and that Apple doesn't seem to think it's flagship photo app. warrants having such functionality.
If I have somehow overlooked a workable solution, please poke me before I begin my arduous trek down the road of mind-numbing photo library editing. ;-)
more edit: I just found this, which also promises to assist, and should be included in our script, IMHO:
http://hints.macworld.com/article.php?story=20060624112253828 Separating out the Original photos from the Edited photos seems to be a good idea.
or... is Tidy Up now Aperture3 compatible, and the best solution we have?
FWIW, enabling Auto-Stack appears to be broken. Aperture has been spinning gay all day now and nothing seems to be happening.