File Deduplication

Discussion in 'Digital Photography' started by compuwar, Dec 28, 2010.

  1. compuwar macrumors 601

    compuwar

    Joined:
    Oct 5, 2006
    Location:
    Northern/Central VA
    #1
    I've been struggling with deduping my image files for years. It seems every program I find simply does ok until it is presented with a large amount of data, then it fills up memory (8G on my MacPro) and falls over.

    Well, I think I may finally have found the answer to my problem that doesn't involve me spending a week pulling my hair out trying to write my own solution. Singlemizer seems to do the right stuff no matter what.

    http://minimalisticdev.com/Singlemizer/

    It meets my basic functional requirements:

    1. Can filter what it's looking for.
    2. Uses a good hash mechanism to actually compare file data.
    3. Doesn't store everything in memory as it goes along.
    4. Can be purchased with PayPal.
    5. Performance is very good.

    I have no connection with the company other than just having forked out the $20 registration fee despite still being under the 20 free trials.

    It chunked along on 1.1TB of data in a reasonable amount of time, and the system did waaay better than with my last attempts with other software. I was able to dedup an entire hard drive in a single run without crashing or grinding the system to a swapping halt on the very first try.

    The application seems to do the right thing deletion-wise. It organizes everything in the trash by original folder, so undoing things before emptying the trash will be sane if necessary and the directory is date/time named too, so multiple runs can be figured out. This means keeping things like my Zenfolio duplicates can be done post-deletion rather than having to specify the folder name in an exclusion list, though long-term the list is the way to go.

    One license is good for every Mac I own too- it's nice to not have to worry about using it on my laptop drives too- so far I'm a happy camper.

    Paul
     
  2. aristobrat macrumors G4

    Joined:
    Oct 14, 2005
    #2
    Looks awesome, wish there was a version with those features that worked on duplicates in an Aperture library. :confused:
     
  3. compuwar thread starter macrumors 601

    compuwar

    Joined:
    Oct 5, 2006
    Location:
    Northern/Central VA
    #3
    I always use referenced files, I don't trust any database with that data! I've seen way too many corrupt databases in my lifetime...

    Paul
     
  4. firestarter macrumors 603

    firestarter

    Joined:
    Dec 31, 2002
    Location:
    Green and pleasant land
    #4
    How did you wind up with dupes in the first place?
     
  5. compuwar thread starter macrumors 601

    compuwar

    Joined:
    Oct 5, 2006
    Location:
    Northern/Central VA
    #5
    Years and years of copying one hard drive to another to another.

    Paul
     

Share This Page