APFS External Drive Data Loss

[sideshowuniqueuser]

Has anyone else had problems with file changes not flushing to an external SSD formatted in APFS (encrypted)?

I am migrating my files from my old MBP (2015 rMBP, running Mojave) to my new one (2021 M1P MBP, Running Monterey 12.4), and created a "Migrate.rtf" file using TextEdit, simply noting which folders I'd copied or not, and saved it directly to the external SSD. I then ejected the drive, connected it to the new MBP, and discovered the Migrate.rtf was missing. When I opened TextEdit on the new MBP, and did File -> Open Recent -> Migrate.rtf, it said it can't find it on the SSD. I plugged the SSD back into the old one, and tried the same, and tried to search on Finder, and it simply didn't exist anymore. So I recreated it, saved some edits. Made some more changes, and saved it, and ejected the drive again. Plugging into the new MBP, the Migrate.rtf did exist on the external SSD, but the latest edits I made were missing.

WTF is going on? Is the data not flushing correctly to APFS?

 

[Fishrrman]

I'm a neanderthal (take a look at the avatar, it accurately represents me).

Having said that...
I suggest you get paper and pencil and keep a HANDWRITTEN list of stuff, folders and files that have been migrated.

Then you won't have to worry about losing that list...

 

[sideshowuniqueuser]

I'm a neanderthal (take a look at the avatar, it accurately represents me).

Having said that...
I suggest you get paper and pencil and keep a HANDWRITTEN list of stuff, folders and files that have been migrated.

Then you won't have to worry about losing that list...

What, nah, you missed the entire point. The point wasn't at all about the contents of the actual "Migrate.rtf", that was just to illustrate a specific example of a file that I explicitly knew there was lost data when copying to the external drive.

The point is, that if you have an external SSD, and format it to APFS, and then copy files to it, there is absolutely no guarantee that all the files will actually be saved to the external drive. Finder will show them as copied, but when you eject the drive, and then plug it into another Mac, the files might, or might not, be on the drive!

My presumption is that the files aren't flushing, that there is still some of the data sitting in a buffer, showing in Finder as being copied to the external drive, but not actually flushed to the physical drive itself. The reason a drive is supposed to be ejected correctly, rather than just unplugged, is to make sure the OS flushes any buffers first. However, it seems that with macOS (well, at least Monterey and Mojave) and APFS on an external SSD, the buffer is not being flushed when you eject the drive, and thus, some of the data is being lost.

So my question was about asking if anyone else had had the same problem, and if they new a solution to it.

 

[gilby101]

So my question was about asking if anyone else had had the same problem

Nearly all my documents, photos, etc. are on external SSD (APFS encrypted). I have never had your problem.

What SSD? Your problem may be related to how the drive responds to the eject.

 

[sideshowuniqueuser]

Nearly all my documents, photos, etc. are on external SSD (APFS encrypted). I have never had your problem.

What SSD? Your problem may be related to how the drive responds to the eject.

Thanks for that.

I seem to have isolated the problem, and it seems to always be only the last file or folder copied onto the drive that fails to flush and goes missing.

You might be right that it is the drive I'm using. It's the original 500GB drive from a 2015 rMBP that I'd upgraded to 1TB, and put the old 500GB drive into an external enclosure. So yeah, it's 7 years old, and maybe it is just that drive that isn't flushing.

Can you possibly double check for me by copying a few files onto your external SSD, then copy one more file, and eject the drive, then plug it into another Mac and check if the last file copied correctly?

 

[gilby101]

Can you possibly double check for me by copying a few files onto your external SSD, then copy one more file, and eject the drive, then plug it into another Mac and check if the last file copied correctly?

All good for me. Using a Samsung T5. Its volumes are APFS.

Your issues could be:
macOS failing to flush buffers.
The enclosure you are using.
The SSD drive.

I would guess limitations in the enclosure, but I have no evidence for that!

 

[elptdbi3lYI]

Try copying with rsync:
rsync -avP /path/to/file.txt /path/to/destination/

 

[sideshowuniqueuser]

All good for me. Using a Samsung T5. Its volumes are APFS.

Your issues could be:
macOS failing to flush buffers.
The enclosure you are using.
The SSD drive.

I would guess limitations in the enclosure, but I have no evidence for that!

Thanks, appreciate that.

 

[gilby101]

Just something that made me think of your issues. May, or may not, be relevant.
I was reading the latest bits of discussion in https://eclecticlight.co/2022/07/19/what-can-you-do-with-time-machine-backups-on-apfs/. In particular that when a TM volume is unmounted it disappears from the desktop and Finder, but at that instant the APFS container may not have completed its unmounting. In most cases only a very small time window between the two unmounting steps. It made me wonder of you are suffering from a delay between volume and container unmounting. You need to use Disk Utility to confirm the complete unmounting of the disk and container.