Help restoring APFS (High Sierra) dd raw image

[brdeveloper]

Hello,

I've recently sent my Early-2013 rMBP to the assistance for fixing the battery defect ("Service Battery" message and unstable battery life). So I've formatted the SSD after extracting a dump (via dd command and saving the raw image as DMG) from two OSX installations (I had dual boot between Mavericks and High Sierra). When I received my Macbook back, I could promptly restore the Mavericks installation from the "restore" disk utility option from the system recovery mode. However, disk utility seems unable to restore the High Sierra partition. The DMG raw image can be mounted flawlessly from a High Sierra install, I'm just unable to move that to the SSD.

Any ideas?

 

[treekram]

Have you tried reversing the if= and of= parameters from the dd command you used to create the image? (You can use Mavericks or the Terminal in Recovery - unmount the APFS partition before restoring.)

Did you erase the SSD prior to service for privacy and are now restoring it? Did you just erase the partitions but kept the original partition table?

 

[brdeveloper]

Have you tried reversing the if= and of= parameters from the dd command you used to create the image? (You can use Mavericks or the Terminal in Recovery - unmount the APFS partition before restoring.)

I've already tried restoring through dd command, but it seems that it created a corrupted partition.

Did you erase the SSD prior to service for privacy and are now restoring it? Did you just erase the partitions but kept the original partition table?

Yes, for privacy purposes, since I don't encrypt the filesystem. It's just for preventing dummy technicians from stealing data. I've actually erased everything, that is, all the SSD was set as free space. The images were created through "dd if=/dev/diskXsY of=/Volumes/ExternalDrive/HighSierra.dmg" or something alike.

 

[treekram]

I've already tried restoring through dd command, but it seems that it created a corrupted partition.

Yes, for privacy purposes, since I don't encrypt the filesystem. It's just for preventing dummy technicians from stealing data. I've actually erased everything, that is, all the SSD was set as free space. The images were created through "dd if=/dev/diskXsY of=/Volumes/ExternalDrive/HighSierra.dmg" or something alike.

What I have noticed, and others have noticed as well, certain disk imaging functions don't work with regards to APFS containers/volumes in Disk Utility. I looked at the developer betas of Mojave that still seems to be the case.

What I have also noticed is that there seems to be a greater dependency on the various unique ID's used to identify a disk and it's components used in High Sierra (APFS or not). In testing I've done, while doing intelligent copying of files from a system disk to another disk (intelligent being taking into account links, special device files, etc.) used to produce a working, bootable clone up to Sierra, that's no longer the case with High Sierra in some instances - and it doesn't matter if it's APFS or not. So what I'm wondering (and I don't know if Apple has disclosed this or not) is whether or not to restore a dd-produced image of a APFS partition (container) requires that you restore to a partition that has the same "Partition UUID" (and that's why I asked if you still retained the partition table). In the normal *nix world, typically what you've done should work. You should make sure, to the best extent possible, that your disk is partitioned exactly the same as it was before. If that is the case and it's still not working, I think you should call Apple and ask for advice (of course you can search the web as well).

 

[treekram]

Thinking about it - one thing you can try is Disk Utility First-Aid - it would have to be from a bootable High Sierra disk or Disk Utility on a High Sierra installer on a flash drive.

 

[brdeveloper]

What I have noticed, and others have noticed as well, certain disk imaging functions don't work with regards to APFS containers/volumes in Disk Utility. I looked at the developer betas of Mojave that still seems to be the case.

What I have also noticed is that there seems to be a greater dependency on the various unique ID's used to identify a disk and it's components used in High Sierra (APFS or not). In testing I've done, while doing intelligent copying of files from a system disk to another disk (intelligent being taking into account links, special device files, etc.) used to produce a working, bootable clone up to Sierra, that's no longer the case with High Sierra in some instances - and it doesn't matter if it's APFS or not. So what I'm wondering (and I don't know if Apple has disclosed this or not) is whether or not to restore a dd-produced image of a APFS partition (container) requires that you restore to a partition that has the same "Partition UUID" (and that's why I asked if you still retained the partition table). In the normal *nix world, typically what you've done should work. You should make sure, to the best extent possible, that your disk is partitioned exactly the same as it was before. If that is the case and it's still not working, I think you should call Apple and ask for advice (of course you can search the web as well).

Thank you. Perhaps I should have made a complete disk dump of the SSD instead of the macos installations separately. I guess this way I've lost the partition table or something, and the "dd restore" strategy won't work. Regarding partition sizes, the only care I had on restoring was creating a bigger partition than the Mavericks image, since the raw image isn't resized automatically at the restore process. After restoring, I could resize it to a smaller size from disk utility. This would provide a bigger free space than the required to restore the HighSierra image. But it didn't worked. Perhaps I should try creating partitions of the exact same size as the original ones.

 

[Fishrrman]

Is there any chance you could mount the High Sierra disk image, and then "restore" it to an EXTERNAL drive?

If this is successful, and if the external drive is bootable, I would then use CarbonCopyCloner (free to download and use for 30 days) to clone the external drive to the INTERNAL partition you want to use for High Sierra, to see if that works...

 

[Kingcr]

dd if=/dev/diskXsY of=/Volumes/ExternalDrive/HighSierra.dmg" or something alike

Firstly, if you imaged your system disk while running the system you were imaging, you could end up with all sorts of problems.

Secondly, the dd command you posted would only image one partition (or slice in BSD parlance). Assuming you imaged your system partition, you’d be missing your EFI boot partition. So unless you know exactly what you did, you may not be able to just restore your image.

 

[treekram]

Firstly, if you imaged your system disk while running the system you were imaging, you could end up with all sorts of problems.

Secondly, the dd command you posted would only image one partition (or slice in BSD parlance). Assuming you imaged your system partition, you’d be missing your EFI boot partition. So unless you know exactly what you did, you may not be able to just restore your image.

When I try to dd a mounted APFS container, it says "Resource busy" which is what one should expect, so I doubt that the OP imaged a mounted APFS container. As for the second sentence, yes, the OP should have imaged the entire disk as people doing this have been able to restore it, but the EFI partition isn't an issue. It will usually just contain files for staging EFI firmware udpates.

Is there any chance you could mount the High Sierra disk image, and then "restore" it to an EXTERNAL drive?

If this is successful, and if the external drive is bootable, I would then use CarbonCopyCloner (free to download and use for 30 days) to clone the external drive to the INTERNAL partition you want to use for High Sierra, to see if that works...

You actually may be able to use CCC but not in the way described in the post. The people at Bombich seem to understand how APFS partitioning works and how to clone it. If the Disk Utility First-Aid doesn't work, I would restore the APFS container image back to the SSD, again making sure the partitions are the same as the dd image sizes. Then try to clone the entire SSD to another disk at least the size of the SSD. CCC may be able to make the necessary adjustments so that the clone will work even though it's not working on the source SSD - if nothing else is working, it's worth a try. You'd need a clean external disk to clone to, another disk with (likely) High Sierra to boot from and the internal SSD to clone.

 

[brdeveloper]

Firstly, if you imaged your system disk while running the system you were imaging, you could end up with all sorts of problems.

Actually, I've generated each image from the opposite system installation as I had dual boot.

Secondly, the dd command you posted would only image one partition (or slice in BSD parlance). Assuming you imaged your system partition, you’d be missing your EFI boot partition. So unless you know exactly what you did, you may not be able to just restore your image.

I was expecting the "restore" option from ASR disk utility would do the trick. Actually it did when restoring Mavericks dmg. In short: you can dd a non-running system partition and restoring it from disk utility, UNLESS it's an APFS partition. And yes, my mistake was not DD'ing all the SSD in the same dmg, so I could get EFI partition and other stuff outside the single partitions. However, Mavericks seems pretty self-contained since it was restored flawlessly.
[doublepost=1530217781][/doublepost]

When I try to dd a mounted APFS container, it says "Resource busy" which is what one should expect, so I doubt that the OP imaged a mounted APFS container.

Yes, that was the case. I've always imaged the "offline" partition from the online one. I'll try your CCC approach soon.

 

[Kingcr]

Just out of interest, can you post "gpt -r show /dev/disk<x>" for disk containing the APFS partition you attempted to restore to?

 

[brdeveloper]

Just out of interest, can you post "gpt -r show /dev/disk<x>" for disk containing the APFS partition you attempted to restore to?

I'll do it later, but I've already restored the HighSierra dd image through Carbon Copy Cloner (had to mount the image previously). Great piece of software!