Getting a VERIFIED clone of an APFS system disk?

[Denis M.]

Am I the only person who feels that there's a serious (a very big understatement) security problem in Disk Utility since APFS was introduced?

Under HFS+, one can without problem make with DU a VERIFIED clone of a system disk and boot on the clone, knowing it is an exact copy of the system disk used as a source for the cloning operation.

Unless I'm dense (and, if it is the case, I would more than welcome any explanation/clarification/education), while duplicating a data APFS volume with DU works OK, it is not possible to do the same with an APFS SYSTEM disk: when one tries to do it, this fails and the user gets the infamous "APFS failed to invert the volume - Resource busy. The operation couldn't complete. (OSStatus error 16)." message.

Aside from the fact that I would very much like to know the exact meaning of OSStatus error 16 and what on Earth is a volume inversion and why it is needed here (many thanks to Apple for their extensive and widely available technical documentation...), this failure to provide a way to produce a VERIFIED copy of a system disk is simply a way to send any user, sooner or later, to his doom.

Now, I am well aware of the existence of Time Machine and of such third party utilities as, for example, Carbon Copy Cloner or SuperDuper. While I haven't tried SD, I have CCC and use it daily.

I have given up using Time Machine a while ago, the reason being that, while tests I had done prior to putting it to use had worked satisfactorily, I had to use it in real need twice and in both cases, Time Machine failed me. I was only saved because, being a firm believer in belt and suspenders policy, I had other types of backups available. In the first case, the restored system disk turned out not to be bootable (for unknown reasons), while the tests I had previously done had produced bootable clones. No error message had ever been issued by Time Machine either during the save or the restore operations. I was just told: "Well, it may happen"... (side question: would you accept such an answer in the case of a non-functioning airbag in a car?) In the second case, I asked the migration assistant to use a freshly made Time Machine backup (only one run of Time Machine, which produced no error message) as source for its operations during a macOS version upgrade. The migration assistant reported missing HUNDREDS of files... The unescapable conclusion is that Time Machine is, at best, an unnecessarily complicated copy utility, but cannot in ANY circumstances be termed a backup utility. It shouldn't have failed in the first place and, if or when it met with problems, it should have reported them.

CCC, with its "search for damaged files" option, does a good job. Still, it is not a VERIFIED backup and I really wish to have this feature available, even if, short of implementing reliable disk shadowing, it necessarily means making an offline backup of the system disk. I'm ready to trade this penalty for security (I have done it routinely at least every fortnight with HFS+ for the past eleven years).

When is Apple (or anybody else) going to provide a way to produce a VERIFIED APFS system disk clone?

Upgrading production systems to Mojave is out of question for me in the current situation.

Denis M.

 

[posguy99]

How does the above describe a security problem?

 

[Mr. Retrofire]

@Denis M.
TL;DR

Did you try the disk cloning via the recovery partition or USB thumb drive with Mojave?

 

[BLUEDOG314]

So I have been using hdiutil to make restorable images of clean APFS high sierra installs. I have made Bootable installers which I can use to access terminal, allowing me to capture the whole APFS container of a boot volume. Works like a charm but 10.13.4-10.13.6 produces the inversion error you mentioned.

Since Mojave, I've found that I do not receive the error anymore. The hdiutil method produces a verified dmg that can then be imagescanned and restored.

Would this work for you or am I misunderstanding you?

 

[Denis M.]

How does the above describe a security problem?

I wasn't meaning security in the restricted sense of protection against any form of malevolent aggression, but more generally as protection against any form of destruction of or any type of damage to the system disk, be it caused by an accident or an attack.
Denis M.
[doublepost=1538863872][/doublepost]

@Denis M.
TL;DR

Did you try the disk cloning via the recovery partition or USB thumb drive with Mojave?

Not from a USB thumb drive, but both from the recovery partition and from the Disk Utility when booted from another (Mojave too) system disk. Same result in both case.
[doublepost=1538864024][/doublepost]

So I have been using hdiutil to make restorable images of clean APFS high sierra installs. I have made Bootable installers which I can use to access terminal, allowing me to capture the whole APFS container of a boot volume. Works like a charm but 10.13.4-10.13.6 produces the inversion error you mentioned.

Since Mojave, I've found that I do not receive the error anymore. The hdiutil method produces a verified dmg that can then be imagescanned and restored.

Would this work for you or am I misunderstanding you?

Sorry, but I don't happen to know what hdiutil is. Could you, please, provide some infos? It may be a solution to my concern.
Denis M.

 

[DeltaMac]

...
Sorry, but I don't happen to know what hdiutil is. Could you, please, provide some infos? It may be a solution to my concern.
...

hdiutil is a terminal command that allows you to manipulate disk images. It has a wide range of options, making it very versatile. You can do a man hdiutil in the terminal to see a listing of those options.
Maybe BLUEDOG314 will return with some steps that you can use with that.

 

[Denis M.]

hdiutil is a terminal command that allows you to manipulate disk images. It has a wide range of options, making it very versatile. You can do a man hdiutil in the terminal to see a listing of those options.
Maybe BLUEDOG314 will return with some steps that you can use with that.

Thank you. I just had a quick look at the output of "man hdiutil" and it seems I have quite a bit of homework to do before I feel comfortable with this. I noticed one of the verbs was "verify", which may or may not answer my concern, I'll need to check that (note that my concern was for obtaining a VERIFIED clone of the APFS system disk; if I just want to get an unverified clone, CCC will do it nicely for me).
Denis M.

 

[BLUEDOG314]

I believe hdiutil will do the trick. My workflow for getting an image to restore to machines is as follows, but should work for you as well. I use a compressed format which checksums the the image meaning if I open the .dmg it is read only and you get that long "verifying" thing before it opens.

First, you have to have a place to store the image obviously, and you need to boot from a source that is not on the APFS container you want to get an image from. I use an SSD flash drive with a small Mojave installer. The other larger partition is where the image goes.

1. Boot to the installer and open terminal.

2. Find the disk identifier for the APFS container. Run 'diskutil list' and find something like "disk0s2" that corresponds with Apple_APFS Container. This will I'm assuming be on the internal drive. If you need help you can post your terminal output.

3. Create the disk image. I use the compressed format like I mentioned above. 'hdiutil create -srcdevice /dev/disk0s2 -format UDZO /Volumes/PathToWhereYouWantToStoreImage'

At this point you have an image that you can use. The easiest way to restore the image is with the asr command, but you must first imagescan your result.

4. Scan the image with 'asr imagescan --source /Volumes/PathToWhereYouWantToStoreImage'

Again, reading, verifying, imagescanning a large image will take a really long time so having it on an SSD is better if you can. Also, you don't need to actually imagescan unless you are definitely going to restore that particular image.

Now to restore, if you had to. First, erase your target drive with an empty APFS container. This again will likely be the internal "disk0" drive.

5. Erase and prep by booting to your Mojave installer or using another Mac via target disk mode with 'diskutil erasedisk apfs "Macintosh HD" disk0' obviously being careful not to erase the wrong drive.

6. Restore the image via 'asr restore --source /Volumes/PathToWhereYouWantToStoreImage --target /dev/diskAxB --erase' where "diskAxB" refers to the blank APFS container you just made. You can get this by using 'diskutil list' again.

I've done this countless times with success, and while it may look confusing, its quite easy. Maybe if there is confusion I can make a video. Also keep in mind if you try to restore an APFS container to a machine that has not had APFS on it yet, it will not boot as it hasn't received necessary firmware yet. There is an easy solution to that by repacking Apple firmware with pkgutil and applying with the installer terminal command. I'll save those directions unless you need them.

 

[Denis M.]

I believe hdiutil will do the trick. My workflow for getting an image to restore to machines is as follows, but should work for you as well. I use a compressed format which checksums the the image meaning if I open the .dmg it is read only and you get that long "verifying" thing before it opens.

Thank you very much for taking the pain to write all these instructions. I obviously need to practice a lot before I'll be comfortable with all this (my background is not Unix, but VMS, meaning that, while I've a long experience in system support and internals, I'm not really at home with bash and Terminal, although I think I understood most of what you wrote). I'll have to practice and test before I manage to do the trick, but I hope I'll be able to do it.

That said, an average user (which is what I am under macOS) should not have to go down to this level of details. Apple should have given from the beginning in Disk Utility the same function for APFS it provides for HFS+ (the ability to produce a verified clone of a system disk). This is of utmost importance for the security of any system. However, Time Machine is, in itself, sufficient proof that Apple is unable to grasp the major importance of being able to RELIABLY save a system. Depending on what one is doing with his system, this may very well be a matter of life and death (figuratively or even literally).

Thank you again.
Denis M.

 

[BLUEDOG314]

That said, an average user (which is what I am under macOS) should not have to go down to this level of details.

I agree. All of this could be scripted, and I actually put some of this together already. Let me know if you would like any of it.

 

[Denis M.]

I agree. All of this could be scripted, and I actually put some of this together already. Let me know if you would like any of it.

Yes, please, I would be very interested. Aside for being extremely useful, it would also teach me quite a few things about how to work under Terminal, which I direly need to learn.

Denis M.

 

[BLUEDOG314]

Yes, please, I would be very interested.
Denis M.

I sent you a PM. If there's broader interest in this perhaps we can continue in this thread as well.