Photos.app data loss after migrating from High Sierra to Mojave

[MilaM]

After migrating my files from High Sierra (10.13.6) to Mojave (10.14.2) using a Time Machine backup, many original image and video files inside the Photos Library disappeared from my hard drive. Actually I can still see them when I browse them in Photos.app. But exporting the original file does not work and the Photos.app complains that the file is missing. This probably means, that the thumbnails are still in the database and that Photos.app knows of the pictures and videos but can't find them on the file system.

What is strange is that images and videos taken originally with my iPhone seem to be fine. Only images and videos sent to me via WhatsApp, then saved to Photos on my iPhone, and subsequently imported to Photos.app on my Mac, were lost during the migration to Mojave or possibly later due to a bug in Photos.app.

My first thought was, that maybe something went wrong with the database migration from version 3 to version 4. In this case at least the original files should be in the Masters folder inside ~/Pictures/Photos Library.photoslibrary/. But I could not find any of the missing files there.

Fortunately I haven't deleted the migrated account on my old Mac running the lates version fo High Sierra yet. There the Photos Library (version 3.0) is still intact and all the images and videos originating from WhatsApp on iPhone are also present in the Masters folder.

Has anyone else experienced this kind of data loss after migrating a Photos Library from High Sierra to Mojave? Is this a know bug?

Also I'm not sure if I should try to migrate the library again or leave it on High Sierra for the time being?

 

[MilaM]

So, after inspecting the logs of my arq backup I came to the conclusion, that the Photos.app migration of the library must have caused the files to be deleted.

Today after upgrading macOS to 10.14.3 and starting with a new copy of my old photos library from High Sierra, the migration went fine, and all the previously missing images and movies are there again.

I'm not sure what caused this bug. I'm leaving this info here just in case someone else has the same issue.

 

[MilaM]

It happened again. After repairing the library Photos.app deleted 291 files from the masters folder.

This time I checked the logs and found the following error:

10:15:53.365478 +0100    Photos    Import failed during master recovery for path: 2018/09/06/20180301-154852/c3z53698-bdc5-25fa-hhtz-158k8u502a3x.JPG, Error Domain=com.apple.photos.reddwarf.ingest Code=1 "Removed version uuid: ss3jujsDWA+jqk+LuqLPcV because imported master was either removed, or chosen as the candidate master during de-dup" UserInfo={NSLocalizedDescription=Removed version uuid: ss3jujsDWA+jqk+LuqLPcV because imported master was either removed, or chosen as the candidate master during de-dup}

Apparently it's an older bug. If found this tweet from February 2018:

https://twitter.com/gesa/status/960292167083204608

Just leaving it here for Google to pick up.

 

[P_Amsterdam]

It happened again. After repairing the library Photos.app deleted 291 files from the masters folder.

This time I checked the logs and found the following error:

10:15:53.365478 +0100    Photos    Import failed during master recovery for path: 2018/09/06/20180301-154852/c3z53698-bdc5-25fa-hhtz-158k8u502a3x.JPG, Error Domain=com.apple.photos.reddwarf.ingest Code=1 "Removed version uuid: ss3jujsDWA+jqk+LuqLPcV because imported master was either removed, or chosen as the candidate master during de-dup" UserInfo={NSLocalizedDescription=Removed version uuid: ss3jujsDWA+jqk+LuqLPcV because imported master was either removed, or chosen as the candidate master during de-dup}

Apparently it's an older bug. If found this tweet from February 2018:

https://twitter.com/gesa/status/960292167083204608

Just leaving it here for Google to pick up.

Hi, I had the same issue. There is n issue with imported whatsapp photo's. On High Sierra no issues, but after migration to Mojava, I can see the foto's, klik on it, see the foto, but I'm not able to export the foto :-(

 

[MilaM]

Hi, I had the same issue. There is n issue with imported whatsapp photo's. On High Sierra no issues, but after migration to Mojava, I can see the foto's, klik on it, see the foto, but I'm not able to export the foto :-(

Sorry to hear that. In my Family this happened on two Macs. Unfortunately in my case the photos were deleted permanently from the library. There are two solutions:

1. Recover them from a pre Mojave backup.
2. Reimport them from WhatsApp.
3. Make screenshots of the thumbnails, this won't work for videos.

The bug has been fixed in the most recent update to Mojave. If you restore an old photos library where the files have not been yet deleted and launch it in Mojave 10.13.5 it should work fine.

 

[P_Amsterdam]

Sorry to hear that. In my Family this happened on two Macs. Unfortunately in my case the photos were deleted permanently from the library. There are two solutions:

1. Recover them from a pre Mojave backup.
2. Reimport them from WhatsApp.
3. Make screenshots of the thumbnails, this won't work for videos.

The bug has been fixed in the most recent update to Mojave. If you restore an old photos library where the files have not been yet deleted and launch it in Mojave 10.13.5 it should work fine.

Hi, the same here, also on 2 laptops :-(. Fortunately I have also another older laptop with High Sierra and the photo database running, so I can compare my old laptop with my new one where Mojave is installed. But on the laptop from my girlfriend where I recently installed Mojave 10.14.5 and took a copy from the backup disk, there is a GAP of over more then 1000 photo's that are missing. So if it should be working after 10.13.5, then it should also work on 10.14.5... so I'm a bit confused.

Currently I just thinking I making a thinking error.. I just copied the photo library from the last backup.. Just checked the last date on my backup disk. Can it be that I need to set the photo library back via a restore in time machine instead of copying the file? This because I have a lot of backups on disks, but maybe the latest version is missing the pictures because I just copied a incremental of the backups!?


BTW: Thanks for mentioning the recover procedures. I'm now working on it to get it all back.

gr,
Paul

 

[MilaM]

I just copied the last complete copy of the whole Photos Library manually to my Mac running Mojave 10.13.4. But this might not be practical if you added a lot of photos to your library since the last good backup.

It might be possible to just copy the missing files into your current library folder. The catch is, that you have to put them into the right folders, or Photos.app won't find them.

The folder where you would have to put them is "Masters" inside your library folder. That folder contains the "original" files. Thumbnails and edited photos are saved somewhere else. The idea of the "Masters" folder is, that Photos won't touch the files and keep them as they where when you imported them.

 

[posguy99]

Is there a reason why people continue to try to use Photos.app? Someone explain to me how putting your pictures into an opaque file format is a good idea, please.

 

[Honza1]

Is there a reason why people continue to try to use Photos.app? Someone explain to me how putting your pictures into an opaque file format is a good idea, please.

Well, sure, let me oblige:
It is convenient and it works. And works very well with iPhone Photos.app and iPad Photos.app and the rest of family, all using apple devices.

And it is NOT opaque format: If you right click on the Library and select "show package contents", you will find folder with all your original images as they were collected - "Masters".
The problem is, that the above is true if you make sure the computer you are working on has in Photos.app preferences selected "Download originals to this Mac", if you are using iCloud. Then (and only then) you can assume your photos are yours (on your computer) and backup with your computer.

If you store originals in iCloud and something weird happens, I have no idea if and how you can get to your originals. iCloud is great tool, but really bad "backup" solution. It is hard to trust it.

FYI: I copy these originals from there as needed and drop them onto my network drive, where they can get picked up and dropped in Picasa which my wife prefers.

Now, why OP had troubles he/she reported, I do not know. Everything worked really well for me during direct in place macOS upgrade - and actually, Photos created new converted Library and left the old one untouched, so loss of data was impossible.

 

[posguy99]

If you store originals in iCloud and something weird happens, I have no idea if and how you can get to your originals. iCloud is great tool, but really bad "backup" solution. It is hard to trust it.

iCloud Photo Library is no sort of a backup solution, and isn't billed as a backup solution by Apple. I find amusing those that try to treat it so.