Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

jmpage2

macrumors 68040
Original poster
Sep 14, 2007
3,230
585
Hey guys I have a somewhat unusual use case and am hoping that I can get some help from people more knowledgeable in this area than myself.

For years I used Lightroom to organize & edit my photos and would periodically do bulk imports of the photos on my iPhone into LR. In addition I had many photos that I took with my SLR camera, imported into LR and then some of those got exported and put on my phone.

I stopped using LR several years ago when they went to a subscription model but wasn't actively managing my photos with my Mac, I just had lots of photos sitting in various places without an overall library in use.

Recently I used a trial to Lightroom to export the entire LR library and then I turned on Apple Photos on my Mac, connected it to my iCloud photo library and also imported the photos I had exported from LR.

As a result, there's some confusion going on (and I really apologize for the lengthy post).

  1. As a result of what I've done I now have many many duplicates. Most of these seem to be copies of photos that are already part of Apple Photos with iCloud and duplicates are ones from the export I did from LR. There is a little ? icon in the corner of the photos that came from the LR export/import I did, is there any reason that is showing up? Is there any preference for one copy of the photo over the other? I chose to download all of my photos to the Mac locally so I assume I have no need to keep both.
  2. When doing imports into the Mac, there is an option to move the photos to the Photos library, what's the preference here? Does it matter as long as the photo library is getting backed up with Time Machine and also exists on iCloud? I can see from an organizational standpoint that leaving the photos in organized folders on my drive and then having them imported into Photos separately provides an extra safety net but seems like this will create many additional copies of photos and get quite confusing to manage.
  3. I have not turned on "optimize storage" because my understanding is that this is a really bad idea since I would have no local copy if the Cloud photos were accidentally deleted. Any further info on this?
Again sorry for the long post, but I have 20,000 pictures and want to make sure I'm managing this library properly.
 

TechSnoop

Suspended
Apr 15, 2022
8
6
The '?' mark in the top right corner of a photo's thumbnail means that it's a missing "referenced photo" i.e. the actual photo isn't stored in the Photos app but somewhere outside, with only its thumbnail in the Photos library and a pointer to its file path in the Photos database. If you try to click open such a photo, you should see this message:

Screen Shot 2022-04-16 at 10.23.15.png


Go to Photos Preferences - General, and see if Importing - Copy items to the Photos Library is unchecked. If yes, check mark it. Then create a smart album with 2 conditions: <Photo><is><Referenced> and <Date added><is><Date> (In the latter condition, Date should be when you imported photos from LR). Presumably all items in this smart album will have the '?' mark. If so, delete them all. After this, re-import the photos from LR into Photos.

If you have enough space on your Mac to store all your original photos, you can uncheck 'Optimize Mac Storage'. If not enough space (but you do have enough storage available in your iCloud account), check mark it.
 
  • Like
Reactions: webbga

jmpage2

macrumors 68040
Original poster
Sep 14, 2007
3,230
585
The '?' mark in the top right corner of a photo's thumbnail means that it's a missing "referenced photo" i.e. the actual photo isn't stored in the Photos app but somewhere outside, with only its thumbnail in the Photos library and a pointer to its file path in the Photos database. If you try to click open such a photo, you should see this message:

View attachment 1992166

Go to Photos Preferences - General, and see if Importing - Copy items to the Photos Library is unchecked. If yes, check mark it. Then create a smart album with 2 conditions: <Photo><is><Referenced> and <Date added><is><Date> (In the latter condition, Date should be when you imported photos from LR). Presumably all items in this smart album will have the '?' mark. If so, delete them all. After this, re-import the photos from LR into Photos.

If you have enough space on your Mac to store all your original photos, you can uncheck 'Optimize Mac Storage'. If not enough space (but you do have enough storage available in your iCloud account), check mark it.

Thanks, this is very helpful.

I did not have Copy items to the Photos Library checked. I believe I read an article several months ago that argued you should leave your original photos in place and not move them into Photos.

When I open the files with the ? I do not see the error message you indicated, the file opens. I wonder if this is because as long as I am opening it on the Mac and the files are accessible it is able to locate them.

A bit disconcerting as I have been deleting duplicates and it was not clear what was going on, it seems now I will have to do a bit more work. My concern is that when I re-do the import (which is quite large) I will end up with some "new" duplicates of the import where I have both the imported version and the version that was already in iCloud. Without the "?" I might not be clear on what is going on (am I deleting the full resolution file or a smaller version?). It's too bad that Photos can't detect the duplicates, Lightroom if I remember correctly would do this.

In general is it recommended to copy items to the Library but then also keep the originals as a backup?
 

TechSnoop

Suspended
Apr 15, 2022
8
6
Thanks, this is very helpful.

I did not have Copy items to the Photos Library checked. I believe I read an article several months ago that argued you should leave your original photos in place and not move them into Photos.

When I open the files with the ? I do not see the error message you indicated, the file opens. I wonder if this is because as long as I am opening it on the Mac and the files are accessible it is able to locate them.

A bit disconcerting as I have been deleting duplicates and it was not clear what was going on, it seems now I will have to do a bit more work. My concern is that when I re-do the import (which is quite large) I will end up with some "new" duplicates of the import where I have both the imported version and the version that was already in iCloud. Without the "?" I might not be clear on what is going on (am I deleting the full resolution file or a smaller version?). It's too bad that Photos can't detect the duplicates, Lightroom if I remember correctly would do this.

In general is it recommended to copy items to the Library but then also keep the originals as a backup?
Importing with "Copy items to the Photos library" checked copies them into the Photos library. Original photos also stay at the source location, so you can back them up off the Mac in regular folders. An excellent summary of how Apple Photos works can be found here - it's well worth a read.

Referenced files are flaky and best avoided. If you can open files with "?" notation, it may be because those broken links were created when Photos could not access those files stored outside the Photos library but now it can access them. If you find all referenced items imported from LR as explained in my previous note, and delete them, then re-importing won't duplicate these items. You may still duplicate items that are in LR and also already in iCloud - hard to say without knowing the full history.
 

jmpage2

macrumors 68040
Original poster
Sep 14, 2007
3,230
585
I will read that material, thanks for linking it.

The scenario I'm kind of dreading is that since I didn't fully understand why I had the "?" photos but had so many duplicates after my import I deleted most of those files, but not all of them.

If I do a new import I will end up with duplicates again.

It's something I will solve on my own, your info has been really helpful, thank you.

Out of curiosity do you know how iCloud deals with duplication and deduplication between multiple devices and accounts?

I have photos showing up that also show up with a 2nd copy that was sent by my wife, and then I'm sure she has saved photos that I sent her that are in her account.

Are all three photos going to be separately backed up on iCloud or will it recognize that the files have the same checksum and only keep one copy? Similarly I wonder how it is handled when multiple devices that have the photos stored locally get backed up to iCloud and also have duplicates of the library.

Thanks again.
 

ForkHandles

macrumors 6502
Jun 8, 2012
471
1,173
I will read that material, thanks for linking it.

The scenario I'm kind of dreading is that since I didn't fully understand why I had the "?" photos but had so many duplicates after my import I deleted most of those files, but not all of them.

If I do a new import I will end up with duplicates again.

It's something I will solve on my own, your info has been really helpful, thank you.

Out of curiosity do you know how iCloud deals with duplication and deduplication between multiple devices and accounts?

I have photos showing up that also show up with a 2nd copy that was sent by my wife, and then I'm sure she has saved photos that I sent her that are in her account.

Are all three photos going to be separately backed up on iCloud or will it recognize that the files have the same checksum and only keep one copy? Similarly I wonder how it is handled when multiple devices that have the photos stored locally get backed up to iCloud and also have duplicates of the library.

Thanks again.
Two things
In your ’Pictures’ directory lives your Photos file. right clicking this gives you a show package contents option, choose this. You will then see a folder called originals, where all your originals exist. So you don’t need another set of folders on your hats drive with originals, transfer those to a back up drive for safe keeping.

The search function is also very good. You can identify all photos by camera. This will allow you to find and place into smart albums which photos came from which source.
for example. If iPhone photos were king you could find them all, tag them as favourite, and then bee be able to discern which of two duplicates you wanted to keep.
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.