Organizing a very large and disorganized photo collection

[zqbobs]

I have an external SSD with about 700K photos (about 1.4TB), including many duplicates. Unfortunately, I have not been careful and consistent enough in organization over the years, and the photos are scattered in numerous folders, sometimes nested rather deep. Many have Finder tags, but not all; file and folder naming is only semi-consistent, etc. They are a broad mix of scanned film photos (hence with no EXIF dates to help) and more modern digital photos.

I want to designate one primary folder (PF) to reorganize photos properly and have started that process using chronology as the main organization framework. I need advice on how to proceed efficiently - should I first eliminate duplicates in the PF and then do the same on the entire collection? Should I work on the rest (outside the PF), folder by folder, and then work on all at once to find duplicates across different folders? Should I start with the deepest nested folders and work outward to include higher level folders?

I’ve been using Duplicate File Finder (DFF) Pro to find duplicates. It’s thoroughness and accuracy is impressive, and it has a large set of rules to control how duplicate searches are made, but if anyone has other app recommendations, I’m interested. I have also at times used PhotoSweeper, Gemini and Duplicate Detective, but none seem as good to me as DFF. Where I most need help is in designing an efficient plan.

Any tips on a general approach to this problem will be welcome!

 

[zqbobs]

OK, 320 views and no comments - maybe in the wrong thread? Or, maybe because it’s a Finder-specific problem, not Photos. In any case, here’s what I’m doing for now, working from the bottom up in my Finder folder structure (aka “mess”):

1. assigning dates to *all* photos as best I can, i.e. YYYY-MM-DD in front of file name, or YYYY if only that is known, or YYYYs, like 1980s, if only a guess.

2. putting photos into decade pots, e.g. 1970s, 1980s, etc.

3. will run Duplicate File Finder on each decade batch separately once dating is done.

I had been reasonably good about using dates in photo filenames, but a conversation with ChatGPT reinforced the importance of that in organizing and finding duplicates. So far, feels like a good approach, albeit brute force.

Still open for advice/comments...

 

[gilby101]

Still open for advice/comments...

You are doing fine....
Organising is messy and tedious. For scanned photos I do have them named by date along with a something about the content - e.g. place and people. Like you, I organise folders by date - year and, when known, month.
Assuming you have Spotlight indexing enabled, it will index the content (e.g. you can search in Finder for "waterfalls"). Not 100% accurate but does help.

 

[Fishrrman]

There are some jobs one just has to do "by hand".
If it's a BIG job, it will take time...

 

[Approved]

https://forums.macrumors.com/tags/

Search for tag name photo organizer

 

[ChrisA]

OK, 320 views and no comments - maybe in the wrong thread? Or, maybe because it’s a Finder-specific problem, not Photos. In any case, here’s what I’m doing for now, working from the bottom up in my Finder folder structure (aka “mess”):

1. assigning dates to *all* photos as best I can, i.e. YYYY-MM-DD in front of file name, or YYYY if only that is known, or YYYYs, like 1980s, if only a guess.

2. putting photos into decade pots, e.g. 1970s, 1980s, etc.

3. will run Duplicate File Finder on each decade batch separately once dating is done.

I had been reasonably good about using dates in photo filenames, but a conversation with ChatGPT reinforced the importance of that in organizing and finding duplicates. So far, feels like a good approach, albeit brute force.

Still open for advice/comments...

The very first thing to do is make a backup copy of this and place the backup in a safe place. Really, you should make two copies and put them in different places and not touch the backup until after you are finished.



Next, I think you could run the duplicate file finders. But why in the betches? Just let it run overnight if it takes that long.



Lastly, place them in folders by year, or year and month if there are many (hundreds) of files in a year.



This is as good as you can do by simply moving files around. Do not worry about filenames, just leave them as they are.



NEXT. This is the biggest part. You need to find some software fr managing photos. Apple's "Photos" app can work. But maybe there is another you like better. I'm using Adobe software.



This is going to take months... You need to go through each photo, one by one, and apply keywords and tags and a zero to five-star rating system, and those photos with no date, apply a date.



The details of how to do this depend on the software you select. Apple's software is easy to use, but then you are committed to Apple for life. Adobe is better, but then you are stuck with that. What software to use is important. So ask this is a separate post, here and maybe in Reddit where there are many more people.

 

[berb]

There's no easy answer. Before macOS Catalina I would have recommended iView MediaPro. Sadly it died along with other 32 bit software, and nothing comes close. Everyone has since moved on to programs that manage and process raw images, such as Adobe Lightroom.

Neofinder is a good alternative to iView, just not as intuitive.

Neofinder can create a catalog of media files, and lets you manage them. You can batch rename based on date created plus a sequence number. The goal is to have unique filenames. Neofinder can also find duplicates. It's also very cheap at $39.95 and includes a 30 day trial.

The DAM Book is the book on image collection management, but not for the faint hearted. Author Peter Krogh established best practice for managing images decades ago, including the 3,2,1 back up strategy.

 

[zqbobs]

Thanks all for your encouragement (needed!) and tips. "The DAM Book" in particular looks interesting, and thanks for the “Tags/photo organizer” forums link. I use NeoFinder daily for everything and have used Finder tags liberally, but not always. When finished with dating all photos, I will take on tagging more consistently. I’ve always wanted an app that would display a window with all my tags (about 100) in a list, that could be simply clicked on to add a specific tag or group of tags to selected photos in Finder, but am not aware of one. I drag photo icons to the sidebar to set tags, which of course works, but is a pain. Does anyone know of a simpler way to tag?

Another very useful app in this area is PicArrange. Really an amazing app with, among other tools, an impressive text-based content search. But, getting ahead of myself - first have to date!

 

[gilby101]

I drag photo icons to the sidebar to set tags, which of course works, but is a pain. Does anyone know of a simpler way to tag?

I select the files in Finder, right click (or Control-click), choose Tags..., and click on the tags I want. Much the same as you do.

 

[zqbobs]

I select the files in Finder, right click (or Control-click), choose Tags..., and click on the tags I want. Much the same as you do.

Yes, a different feel to doing it that way. Better I would say for photos that have the same set of multiple tags - all done at once instead of multiple drags to the sidebar for one tag at a time! Will try to get in the habit of using this procedure more often.

 

[ignatius345]

There's no easy answer. Before macOS Catalina I would have recommended iView MediaPro. Sadly it died along with other 32 bit software, and nothing comes close.

It's been many years since I last used it, but that was a great product.

 

[fatTribble]

In my family only my Dad and I took photos. I became the keeper of the photos and ended up scanning everything on and off over many years.

I think you’re already considering all the right things and others have offered great suggestions.

For what it’s worth, I have all of mine on my MacBook. I did copy maybe a thousand to the Photos app to take advantage of facial recognition so I could find pictures of people quickly and just to have my own favorites readily accessible. I have a tag in Finder called Added to Photos just so I don’t go back a year later and think I should add it again.

Here are just a few of my lessons learned that may or may not be useful:

Go for the easy stuff first like trip photos or events. Trips are well suited for Albums.
My Dad’s Navy photos were easy to pull out too, as an example. Family vacations and holidays were normally already together. Basically anything that naturally belong together. For some things the date isn’t all that important at least to me. So if we had an Album for the time we went to…then that was enough.

By decade is good especially for random photos. I went even broader and had pictures of my parents and grandparents before my siblings and I came along, another for when we were kids and one for when we started moving out.

I think it’s good to consider your audience. Who does this need to make sense to. I didn’t pull out photos of my siblings for the benefit of their kids, for example. They’re all together with our generation.

If there is a way to mark photos as favorites or best then do that from the start as you’re organizing.

I viewed the exercise as creating a static collection. I had no intention of ever adding to my Albums. Most photos now are on iPhone and have plenty of ways to find them. And my adult siblings don’t care about my new photos.

Go for good. Perfection is way too much work and probably never really achievable. I don’t know how much my siblings ever looked through the copies I gave them but they did acknowledge the work. Just don’t stress yourself.

Backups.

 

[zqbobs]

In my family only my Dad and I took photos. I became the keeper of the photos and ended up scanning everything on and off over many years.

I think you’re already considering all the right things and others have offered great suggestions.

For what it’s worth, I have all of mine on my MacBook. I did copy maybe a thousand to the Photos app to take advantage of facial recognition so I could find pictures of people quickly and just to have my own favorites readily accessible. I have a tag in Finder called Added to Photos just so I don’t go back a year later and think I should add it again.

Here are just a few of my lessons learned that may or may not be useful:

Go for the easy stuff first like trip photos or events. Trips are well suited for Albums.
My Dad’s Navy photos were easy to pull out too, as an example. Family vacations and holidays were normally already together. Basically anything that naturally belong together. For some things the date isn’t all that important at least to me. So if we had an Album for the time we went to…then that was enough.

By decade is good especially for random photos. I went even broader and had pictures of my parents and grandparents before my siblings and I came along, another for when we were kids and one for when we started moving out.

I think it’s good to consider your audience. Who does this need to make sense to. I didn’t pull out photos of my siblings for the benefit of their kids, for example. They’re all together with our generation.

If there is a way to mark photos as favorites or best then do that from the start as you’re organizing.

I viewed the exercise as creating a static collection. I had no intention of ever adding to my Albums. Most photos now are on iPhone and have plenty of ways to find them. And my adult siblings don’t care about my new photos.

Go for good. Perfection is way too much work and probably never really achievable. I don’t know how much my siblings ever looked through the copies I gave them but they did acknowledge the work. Just don’t stress yourself.

Backups.

I really like all your practical and realistic comments! The idea of doing facial recognition via Photos has occurred to me, but I haven’t followed through. There are some other routes, e.g. GraphicConverter has that capability directly from Finder files, but I haven’t tried that either. A missing link IMO is automatic assignment of Finder tags based on facial recognition - do you or anyone out there have a solution for that? The seems to be a paucity of apps that work with Finder tags.

I don’t have siblings, so this project is for my wife and I, our children and their children, who may not appreciate it at this moment, but I almost guarantee they will when they are over 60!

 

[fatTribble]

A missing link IMO is automatic assignment of Finder tags based on facial recognition - do you or anyone out there have a solution for that?

It’s an interesting idea. Honestly I never went down that path. I basically relied on the folder name and maybe its location in the hierarchy.

Oh, one more thing. I’d suggest first locating every photo you care about. I can’t tell you the number of times I thought I was done only to receive more albums from family members. It felt like it would never end. Oh I’m talking about scanning now not organizing. Sorry.

Eventually I decided I’d finish organizing by a certain date. So I just did the best I could by then.

 

[gilby101]

The seems to be a paucity of apps that work with Finder tags.

There is a nice selection of five apps here https://www.makeuseof.com/mac-apps-that-enhance-finder-tags/. This is a mixture of tag creation and searching by tag.

The first, Leap, was originally developed to use OpenMeta tags. This idea was then incorporated into Mac OS X as Finder Tags. Though I still have a copy, I seldom use it.

I use HoudahSpot which is a front end to Spotlight searching. This is tag aware.

Hazel is great for automatically tagging when files are added to folders.

And at the command line "tag" https://github.com/jdberry/tag/ is invaluable. I use it in scripts. Install with home brew is easiest brew instal tag. It is free.

Fair to say that tagging apps are mostly oriented towards tagging documents. But can equally be used for tagging images.

I have recently discovered Tagception https://madebyevan.com/tagception/. It doesn't do anything you can't do in Finder, but it is fast and could be nice for visualising files (particularly images) with tags.

 

[zqbobs]

There is a nice selection of five apps here https://www.makeuseof.com/mac-apps-that-enhance-finder-tags/. This is a mixture of tag creation and searching by tag.

The first, Leap, was originally developed to use OpenMeta tags. This idea was then incorporated into Mac OS X as Finder Tags. Though I still have a copy, I seldom use it.

I use HoudahSpot which is a front end to Spotlight searching. This is tag aware.

Hazel is great for automatically tagging when files are added to folders.

And at the command line "tag" https://github.com/jdberry/tag/ is invaluable. I use it in scripts. Install with home brew is easiest brew instal tag. It is free.

Fair to say that tagging apps are mostly oriented towards tagging documents. But can equally be used for tagging images.

I have recently discovered Tagception https://madebyevan.com/tagception/. It doesn't do anything you can't do in Finder, but it is fast and could be nice for visualising files (particularly images) with tags.

Thanks for the tips. Tagception looks promising (fast, as you say), but seems a little underdeveloped. HoudahSpot does look useful and has a long history. Going to play with it awhile. I’m familiar with Hazel, but don’t want to move files around to do tagging - want to do it “in place” if possible. I already have enough trouble keeping files organized (see original post)!

 

[BigMcGuire]

So I don't have as many photos as you do - only 62,395 photos and 1,936 videos - about 156 GB. But I found just using Google Photos to be plenty fine - I told it what people it recognized so now I can find anyone and anything by using search. iCloud does the same thing.

iCloud is my primary Photo storage - and Google Photos is my backup. Between them both I can find any photo I'm looking for (Google is a bit better at this) - and sharing large amounts of photos with fiends is super easy with Google.

I realize it's $ but beats sitting down and trying to hand organize imo.

 

[zqbobs]

I am old-school in that I like having direct control of my photos, partly because I don’t trust Google or Apple or… to maintain their ecosystems over a long time. But, in doing so, I know I’m missing out on some convenient features like facial recognition. I also do quite a bit of restoration of old film photos, enhancement of old and new photos, RAW processing, etc. which require interaction and specialized algorithms not available online AFAIK. All of that leads to more files and larger files in some cases.

I do have some photos in both Google and iCloud, and a larger batch in Microsoft OneDrive, but not beyond the basic storage limits. Having 2 cloud services, one primary and one backup, is certainly better than only one, but local backup remains important - see this recent article and comments for example. A good behind-the-scenes way to backup from iCloud is the Photos Backup Anywhere app.

 

[Mike Boreham]

I am old-school in that I like having direct control of my photos, partly because I don’t trust Google or Apple or… to maintain their ecosystems over a long time. But, in doing so, I know I’m missing out on some convenient features like facial recognition.

I also do not trust Apple to keep my photos safe over a long period but I do want the benefits of Photos app.... like Faces and ability to have same photo in several albums (a family group could be in a Holiday or Wedding folder) and AI searching, syncing across devices, etc etc.

My solution is to have my 92K pics in Photos, and use Photos as my primary photo management tool, but once a month I export the months pictures as full size jpegs to an external drive. This has basic organisation by years and topic. So if Apple falls over I have the 92K as regular photos in a folder structure, browsable with Bridge or Graphic Converter for example. When my family pick up the pieces after my demise (not far away as I am 78), this may be more useful than the Photos Library.

I am also in the process of of scanning our multi thousand old prints (pre 2000) and I import these to Photos and put them in year and subject albums. The scanner I am using (Epson FF-680W) gives the file names I assign. But Photos cannot sort on Filename. It can only sort on Title. But there is a very useful app, Photos Workbench, which amongst many other things will batch assign the Title to the Filename.

Incidentally I am delighted with the Epson FF-680W, which I nearly didn't get because of the mixed reviews. It is ideal for the purpose of digitising very large number of prints rapidly with a quality appropriate to the quality of the original material, and also does a very good job automatically restoring badly faded ones. I have high quality flatbed and film scanners for when quality is important but there is no way I will scan 0000s of prints with them. I believe the bad reviews are by people who are making pixel peeping comparisons with specialist scanners, and misunderstand the design philosophy of the FF-680W.