How to fix huge duplicate collection of 68GB?

Discussion in 'Digital Photography' started by deckerinn, Apr 22, 2012.

  1. deckerinn macrumors newbie

    Apr 15, 2012
    Dear members.

    I have 68 GB or 20-30 tsd. photos of my 3 children since 1997, landscape, travelling for an example but I have it on a hard disk as I am afraid of importing it into iPhoto or aperture due to duplicates. I have tried some duplicate programs but they have trouble with processing as many files have exactly same filesize which happens with a lot of photos but they are not always the same. My goal is to create events as I can but all photos have filename of img00568 or numbers only. Does iPhoto or Aperture have options or possibilities to re-organize the library with date taken, place and etc even though I have transferred the collection multiple times between hard disks?

    Questions are therefor:

    1. Which program would you recommend me to import the whole collection even though there are many duplicates?
    2. Would you recommend me to use some duplicate remover before importing it into the recommended photo software as asked in question nr. 1?
    3. Is there available software which can automatically process big libraries with place photo is taken, date and etc.?
    4. Is there a specific process with multiple software which would be the best system and which solutions are those?

    Please feel free to advice me:):confused:

    Br, Deckerinn
  2. milbournosphere macrumors 6502a


    Mar 3, 2009
    San Diego, CA
    My understanding is that iPhoto will use EXIF data to figure out dates, locations, etc., so you might be alright if this data is already present in the images. I know you can sort by location, events, etc. in iPhoto, so perhaps that would help you to categorize your pictures into events. To sort by date: iPhoto> View -> Sort Photos. Included in the drop-down menu is Date.

    As for duplication removal: I played with this software a while ago and it seemed to be pretty good. They have an iPhoto as well as an aperture version. I dunno how large your library is, but some of their testimonials have rather large picture numbers referenced in them, so you might be okay. I'd suggest watching the video demo and watching the video demo to see if it fits your needs. Best of luck!
  3. dba7dba macrumors 6502

    Oct 16, 2008
    Near Apple
    Seems like your problem of having duplicate photos is a very common issue.

    I once helped someone with de-duplicating photos on a Windows 7 PC. He literally had thousands of photos (many duplicates) and he had no idea where to start. I googled with him at his pc and found a free windows tool for de-duplicating photos. In his 1st attempt with the software he made the mistake of trying to de-duplicate too many photos at once and his laptop froze. Eventually it worked out for him.

    However I don't recommend using a de-duplicating software.

    Here's what I recommend. This process really works only if you had set correct date/time on your digital cameras.

    Backup all photos into a completely separate folder before doing anything.
    So let's say you have 'My Pictures' on your HD, just copy it into 'My Pictures-20120422'. Of course this will take a while to complete but this is a must.
    Check 'Info' on both folders to make sure all pictures got copied.

    Download ExifRenamer ( It's a great free tool that renames all photos/video-clips to date/time taken, based on the EXIF data. You install it, open it, and drag/drop the folder with photos/videos in it. This will rename all the files to the data/time taken, (ex.: 2012-01-26_17-26-18.jpg).

    Play around with ExifTool and get familiar with renaming/moving photos.
    -Grab a small sample folder of photos/videos and drop onto icon of ExifRenamer. Obviously you should do a test run to be familiar before working on the actual photos.
    -Also play with the 'Advanced Options' and try moving renamed files to a new folder automatically. I have 1 folder for each day

    Using ExifTool start de-duplicating photos in small batches and also make sure the renamed photos are moved into separate folders. As your comfort level grows, you can start renaming more files at once.

    Once you've renamed all the photos, I recommend organizing the files in this fashion:


    After renaming/orgazing-folders is done, you can import the photos into iPhoto. I also recommend splitting iPhoto library into smaller chunks (1 per year ?) using iPhoto Buddy or iPhoto Library Manager.

    Within iPhoto, you can create smart-albums and create 'events' based on date.

    And lastly, you might want to get into the habit of renaming all new photos downloaded onto your PC with ExifRenamer before doing anything (import, email etc)

Share This Page