APFS: restore to new volume not working

Discussion in 'macOS High Sierra (10.13)' started by chucker23n1, Mar 12, 2018.

  1. chucker23n1 macrumors 68020

    chucker23n1

    Joined:
    Dec 7, 2014
    #1
    So, my situation is (still) that post-High Sierra upgrade, I ended up with three APFS containers, with one volume each. There's still (as of 10.13.4 beta) no way to merge containers, but I've found there is a way to pull in a volume from one container to another, given enough space: the Restore button in Disk Utility, with the special Restore to new volume checkbox. Cool!

    Except it doesn't work

    It gets busy for a while, and then near what appears to be the finish line, it fails to "invert the target volume", whatever that means.

    So,

    • what is "inverting target volume"?
    • what are the equivalent commands in diskutil/asr/etc. so I can try this step-by-step in Terminal?
    • what resource is busy? Surely it's only talking about the volumes within the same container, but those are all unmounted.
     
  2. BLUEDOG314 macrumors 6502

    BLUEDOG314

    Joined:
    Dec 12, 2015
    #2
    Could you do a diskutil apfs list in terminal and paste results please? Also please let me know what your end goal is. I have received the inversion message before, but to my knowledge asr with APFS can only work on entire containers, and requires in image that has been "imagescanned" as well.
     
  3. chucker23n1 thread starter macrumors 68020

    chucker23n1

    Joined:
    Dec 7, 2014
    #3
    Sure.

    Code:
    APFS Containers (5 found)
    |
    +-- Container disk1 F1100A99-B7DD-4112-A8FD-1066924DDA92
    |   ====================================================
    |   APFS Container Reference:     disk1
    |   Size (Capacity Ceiling):      29062860800 B (29.1 GB)
    |   Minimum Size:                 28052054016 B (28.1 GB)
    |   Capacity In Use By Volumes:   11300425728 B (11.3 GB) (38.9% used)
    |   Capacity Not Allocated:       17762435072 B (17.8 GB) (61.1% free)
    |   |
    |   +-< Physical Store disk0s3 D4752035-5A6E-4868-AAE6-CC423D277FFC
    |   |   -----------------------------------------------------------
    |   |   APFS Physical Store Disk:   disk0s3
    |   |   Size:                       29062860800 B (29.1 GB)
    |   |
    |   +-> Volume disk1s1 213000D4-263A-3DD7-89CC-24853BAFF15A
    |       ---------------------------------------------------
    |       APFS Volume Disk (Role):   disk1s1 (No specific role)
    |       Name:                      Détresse (Case-insensitive)
    |       Mount Point:               Not Mounted
    |       Capacity Consumed:         11230281728 B (11.2 GB)
    |       FileVault:                 No
    |
    +-- Container disk3 C679DF15-10AF-4675-B007-315669F6E160
    |   ====================================================
    |   APFS Container Reference:     disk3
    |   Size (Capacity Ceiling):      100000002048 B (100.0 GB)
    |   Minimum Size:                 100247486464 B (100.2 GB)
    |   Capacity In Use By Volumes:   83856551936 B (83.9 GB) (83.9% used)
    |   Capacity Not Allocated:       16143450112 B (16.1 GB) (16.1% free)
    |   |
    |   +-< Physical Store disk0s4 0370D37C-6B6D-4A03-830B-061EE0083ABD
    |   |   -----------------------------------------------------------
    |   |   APFS Physical Store Disk:   disk0s4
    |   |   Size:                       100000002048 B (100.0 GB)
    |   |
    |   +-> Volume disk3s1 6A0C75D4-508C-3057-A0AF-4E1DE2D449D5
    |   |   ---------------------------------------------------
    |   |   APFS Volume Disk (Role):   disk3s1 (No specific role)
    |   |   Name:                      Renaissance (Case-insensitive)
    |   |   Mount Point:               /
    |   |   Capacity Consumed:         78896635904 B (78.9 GB)
    |   |   FileVault:                 No
    |   |
    |   +-> Volume disk3s2 309ECAED-D48A-4173-B1C1-4A633AF2883B
    |   |   ---------------------------------------------------
    |   |   APFS Volume Disk (Role):   disk3s2 (Preboot)
    |   |   Name:                      Preboot (Case-insensitive)
    |   |   Mount Point:               Not Mounted
    |   |   Capacity Consumed:         22265856 B (22.3 MB)
    |   |   FileVault:                 No
    |   |
    |   +-> Volume disk3s3 7D983E87-0C2E-4425-8E84-5D1F168A1721
    |   |   ---------------------------------------------------
    |   |   APFS Volume Disk (Role):   disk3s3 (Recovery)
    |   |   Name:                      Recovery (Case-insensitive)
    |   |   Mount Point:               Not Mounted
    |   |   Capacity Consumed:         518266880 B (518.3 MB)
    |   |   FileVault:                 No
    |   |
    |   +-> Volume disk3s4 9993BAEA-A837-475C-B454-FBEC2C836374
    |       ---------------------------------------------------
    |       APFS Volume Disk (Role):   disk3s4 (VM)
    |       Name:                      VM (Case-insensitive)
    |       Mount Point:               /private/var/vm
    |       Capacity Consumed:         4295397376 B (4.3 GB)
    |       FileVault:                 No
    |
    +-- Container disk2 349D36D5-7188-4B1A-A5AA-12A7BD0C3A78
    |   ====================================================
    |   APFS Container Reference:     disk2
    |   Size (Capacity Ceiling):      371005173760 B (371.0 GB)
    |   Minimum Size:                 356178317312 B (356.2 GB)
    |   Capacity In Use By Volumes:   271784476672 B (271.8 GB) (73.3% used)
    |   Capacity Not Allocated:       99220697088 B (99.2 GB) (26.7% free)
    |   |
    |   +-< Physical Store disk0s2 B3946DEB-341F-4C16-B601-9698BEB3EA00
    |   |   -----------------------------------------------------------
    |   |   APFS Physical Store Disk:   disk0s2
    |   |   Size:                       371005173760 B (371.0 GB)
    |   |
    |   +-> Volume disk2s1 F9B87FA7-D7E6-4C6D-B9F1-C6BE0B6332A1
    |       ---------------------------------------------------
    |       APFS Volume Disk (Role):   disk2s1 (No specific role)
    |       Name:                      Naviguer (Case-insensitive)
    |       Mount Point:               /Volumes/Naviguer
    |       Capacity Consumed:         271652421632 B (271.7 GB)
    |       FileVault:                 No
    |
    +-- Container disk5 F1100A99-B7DD-4112-A8FD-1066924DDA92
    |   ====================================================
    |   APFS Container Reference:     disk5
    |   Size (Capacity Ceiling):      29062860800 B (29.1 GB)
    |   Capacity In Use By Volumes:   11300425728 B (11.3 GB) (38.9% used)
    |   Capacity Not Allocated:       17762435072 B (17.8 GB) (61.1% free)
    |   |
    |   +-< Physical Store ERROR -69629
    |   |   ---------------------------
    |   |   APFS Physical Store Disk:   disk4
    |   |   Size:                       29062860800 B (29.1 GB)
    |   |
    |   +-> Volume disk5s1 213000D4-263A-3DD7-89CC-24853BAFF15A
    |       ---------------------------------------------------
    |       APFS Volume Disk (Role):   disk5s1 (No specific role)
    |       Name:                      Détresse (Case-insensitive)
    |       Mount Point:               Not Mounted
    |       Capacity Consumed:         11230281728 B (11.2 GB)
    |       FileVault:                 No
    |
    +-- Container disk8 F1100A99-B7DD-4112-A8FD-1066924DDA92
        ====================================================
        APFS Container Reference:     disk8
        Size (Capacity Ceiling):      29062860800 B (29.1 GB)
        Capacity In Use By Volumes:   11300425728 B (11.3 GB) (38.9% used)
        Capacity Not Allocated:       17762435072 B (17.8 GB) (61.1% free)
        |
        +-< Physical Store ERROR -69629
        |   ---------------------------
        |   APFS Physical Store Disk:   disk7
        |   Size:                       29062860800 B (29.1 GB)
        |
        +-> Volume disk8s1 213000D4-263A-3DD7-89CC-24853BAFF15A
            ---------------------------------------------------
            APFS Volume Disk (Role):   disk8s1 (No specific role)
            Name:                      Détresse (Case-insensitive)
            Mount Point:               Not Mounted
            Capacity Consumed:         11230281728 B (11.2 GB)
            FileVault:                 No
    Renaissance is the current boot volume.
    Détresse is also a bootable system, but not currently running.
    Naviguer is purely data.

    They're all on their own container, which wasn't intended.

    There's three containers with one volume each. They're all on the same physical disk. I want to turn them into one container with three volumes, to gain the space sharing feature.

    I understand if this isn't possible with the boot volume, but I can't even get the two non-boot volumes to merge.

    The UI in Disk Utility seems to suggest you can take an APFS volume from container X and add it as a new volume to container Y. (Presumably, to accomplish this, it really does multiple internal steps, such as maybe creating a temporary disk image?)

    The asr manpage, then, suggests you can similarly take a single APFS volume from a container disk image and add that to an existing container:

    Code:
         Volume Restore with Creation
              If the target is a synthesized APFS whole disk or Apple_APFS partition, and the --erase option is
              not present, then asr will create a new volume in the given container, after which it will do a
              volume restore to that new volume, as with the previous section.  All other volumes in the con-
              tainer are preserved.
    And:

    Code:
         Creating a new APFS volume on the fly
              sudo asr restore -s <APFS image> --sourcevolumename SourceVolume -t /dev/disk2
        Here we get the same effect as the last example, except that asr will create a new volume on the target
        APFS container disk, given by /dev/disk2, and use that newly created volume as the target.  Any volumes
        which already existed in the container will still be there after the restore.
     
  4. BLUEDOG314, Mar 13, 2018
    Last edited: Mar 13, 2018

    BLUEDOG314 macrumors 6502

    BLUEDOG314

    Joined:
    Dec 12, 2015
    #4
    I have a spare machine that I am going to set up similar to yours in terms of the partitions and see if i can achieve what you want. If it works I will post the entire input to terminal here.

    EDIT: Actually can you tell me exactly what volumes, in terms of the disk identifiers from what you posted, you want in what containers ideally? It looks like multiple volumes have the same name.

    EDIT 2: Ok looking at this one more time what you pasted shows 5 APFS containers so I am a little confused unless this is an error. I am assuming the two at the end showing the error are sort of made up that got their by some failed attempt to fix the issue. I'll post back, I'm setting up a machine like yours now.

    EDIT Final: So I was playing around and attempted using asr with the --restorenewvolume thing and it outright didnt work saying file copy is no longer supported. Now clearly I didnt fill the partitions up as much as you, which brings up the other problem that you barely have enough free space to move things around even if it did work. Then you would have two more issues, one being that you would have to move everything into a container that is not the first container on the disk, and the second being that you would very likely have boot issues if you try to move around the main startup volume and ancillary preboot, recovery, and vm.

    The only other way I can thing of doing this would be to attempt making images of all three containers, imagescanning them and let them reside on an external drive. Then wipe the drive to only one apfs container, first restore the container with the main boot volume, then try using asr and the --restorenewvolume thing again since you now are doing it with an image and not as a "file copy."

    Essentially I do not think you are getting around reformatting without extensive work and image creation which will likely take longer than just backing up contents of data volume and reformatting anyway. If its any consolation, you arent even supposed to be able to have multiple containers on the same physical disk, it straight up blocks creation through terminal. To get it I had to make two hfs+ partitions and convert them afterwards.
     
  5. chucker23n1 thread starter macrumors 68020

    chucker23n1

    Joined:
    Dec 7, 2014
    #5
    Oh wow, you're right. Disk Utility never showed those in the GUI. After a reboot, they disappeared again. Here's a listing that makes more sense to me:

    Code:
    APFS Containers (3 found)
    |
    +-- Container disk1 F1100A99-B7DD-4112-A8FD-1066924DDA92
    |   ====================================================
    |   APFS Container Reference:     disk1
    |   Size (Capacity Ceiling):      29062860800 B (29.1 GB)
    |   Minimum Size:                 28052054016 B (28.1 GB)
    |   Capacity In Use By Volumes:   11308638208 B (11.3 GB) (38.9% used)
    |   Capacity Not Allocated:       17754222592 B (17.8 GB) (61.1% free)
    |   |
    |   +-< Physical Store disk0s3 D4752035-5A6E-4868-AAE6-CC423D277FFC
    |   |   -----------------------------------------------------------
    |   |   APFS Physical Store Disk:   disk0s3
    |   |   Size:                       29062860800 B (29.1 GB)
    |   |
    |   +-> Volume disk1s1 213000D4-263A-3DD7-89CC-24853BAFF15A
    |       ---------------------------------------------------
    |       APFS Volume Disk (Role):   disk1s1 (No specific role)
    |       Name:                      Détresse (Case-insensitive)
    |       Mount Point:               /Volumes/Détresse
    |       Capacity Consumed:         11238498304 B (11.2 GB)
    |       FileVault:                 No
    |
    +-- Container disk2 349D36D5-7188-4B1A-A5AA-12A7BD0C3A78
    |   ====================================================
    |   APFS Container Reference:     disk2
    |   Size (Capacity Ceiling):      371005173760 B (371.0 GB)
    |   Minimum Size:                 371275542528 B (371.3 GB)
    |   Capacity In Use By Volumes:   252938465280 B (252.9 GB) (68.2% used)
    |   Capacity Not Allocated:       118066708480 B (118.1 GB) (31.8% free)
    |   |
    |   +-< Physical Store disk0s2 B3946DEB-341F-4C16-B601-9698BEB3EA00
    |   |   -----------------------------------------------------------
    |   |   APFS Physical Store Disk:   disk0s2
    |   |   Size:                       371005173760 B (371.0 GB)
    |   |
    |   +-> Volume disk2s1 F9B87FA7-D7E6-4C6D-B9F1-C6BE0B6332A1
    |       ---------------------------------------------------
    |       APFS Volume Disk (Role):   disk2s1 (No specific role)
    |       Name:                      Naviguer (Case-insensitive)
    |       Mount Point:               /Volumes/Naviguer
    |       Capacity Consumed:         252789538816 B (252.8 GB)
    |       FileVault:                 No
    |
    +-- Container disk3 C679DF15-10AF-4675-B007-315669F6E160
        ====================================================
        APFS Container Reference:     disk3
        Size (Capacity Ceiling):      100000002048 B (100.0 GB)
        Secondary Minimum Size:       78881599488 B (disk0s4) (78.9 GB)
        Capacity In Use By Volumes:   76009504768 B (76.0 GB) (76.0% used)
        Capacity Not Allocated:       23990497280 B (24.0 GB) (24.0% free)
        |
        +-< Physical Store disk0s4 0370D37C-6B6D-4A03-830B-061EE0083ABD
        |   -----------------------------------------------------------
        |   APFS Physical Store Disk:   disk0s4
        |   Size:                       100000002048 B (100.0 GB)
        |
        +-> Volume disk3s1 6A0C75D4-508C-3057-A0AF-4E1DE2D449D5
        |   ---------------------------------------------------
        |   APFS Volume Disk (Role):   disk3s1 (No specific role)
        |   Name:                      Renaissance (Case-insensitive)
        |   Mount Point:               /
        |   Capacity Consumed:         73207623680 B (73.2 GB)
        |   FileVault:                 No
        |
        +-> Volume disk3s2 309ECAED-D48A-4173-B1C1-4A633AF2883B
        |   ---------------------------------------------------
        |   APFS Volume Disk (Role):   disk3s2 (Preboot)
        |   Name:                      Preboot (Case-insensitive)
        |   Mount Point:               Not Mounted
        |   Capacity Consumed:         22261760 B (22.3 MB)
        |   FileVault:                 No
        |
        +-> Volume disk3s3 7D983E87-0C2E-4425-8E84-5D1F168A1721
        |   ---------------------------------------------------
        |   APFS Volume Disk (Role):   disk3s3 (Recovery)
        |   Name:                      Recovery (Case-insensitive)
        |   Mount Point:               Not Mounted
        |   Capacity Consumed:         517791744 B (517.8 MB)
        |   FileVault:                 No
        |
        +-> Volume disk3s4 9993BAEA-A837-475C-B454-FBEC2C836374
            ---------------------------------------------------
            APFS Volume Disk (Role):   disk3s4 (VM)
            Name:                      VM (Case-insensitive)
            Mount Point:               /private/var/vm
            Capacity Consumed:         2147659776 B (2.1 GB)
            FileVault:                 No
    
    I'm trying to, as a first step just to get my feet wet, move Détresse (disk1s2) into the same container (disk2) as Naviguer.

    I'll post back, I'm setting up a machine like yours now.

    Well, yes and no. disk2 has 118,07 GB free, and disk1 is only 29,06 GB total, of which only 11,31 are used.

    It does get more complicated with Renaissance, yes. But after the first step, I should have a disk2 with 400,07 GB total and 135,83 GB free. Renaissance is 100 GB total, 14,35 GB of which are free. So moving that into the combined new disk2 as a second step should work?

    (If that second step doesn't work, such as because moving the current boot volume is hard, that's fine.)

    Right. Let's ignore those aspects for now and just merge the two non-booting containers.

    I'll look into that.

    I'm pretty sure I ended up with these three containers because that's what Disk Utility did.
     
  6. dsemf macrumors 6502

    Joined:
    Jul 26, 2014
    #6
    If you use the Partition button to add new APFS volumes, it creates another Container. If you select a container and select Add Volume, the volume is created within the selected container.

    Screen Shot 2018-03-16 at 5.03.21 PM.png

    DS
     
  7. chucker23n1 thread starter macrumors 68020

    chucker23n1

    Joined:
    Dec 7, 2014
    #7
    I didn't manually create partitions or volumes; I converted from HFS+ to APFS and incorrectly assumed Disk Utility would be smart enough to convert into an existing APFS container; instead, it created a new one each time.
     
  8. teskor macrumors newbie

    Joined:
    Jan 1, 2013
    #8
    I found this thread by googling, because I had the exact same problem. For reference, I found a workaround and
    posted it here: https://discussions.apple.com/thread/250029045.
    The relevant part of that message is:
    "The process of cloning an apfs volume with disk utility led to the error in the title and left
    me with a volume with only one file "ContainerToInvert" in it.
    I did some googling and found that there's an utility apfs_invert. So I played around and found
    that this command would make the copy work:

    sudo /System/Library/Filesystems/apfs.fs/Contents/Resources/apfs_invert -d /dev/disk3s3 -n ContainerToInvert -s 1

    (where /dev/disk3s3 is the device of the container partition (check with "diskutil list" or "diskutil apfs list" for the right name).

    Is it possible that one can circumvent the error that easily? If yes, couldn't Apple simply fix Disk Utility?"
     
  9. chucker23n1 thread starter macrumors 68020

    chucker23n1

    Joined:
    Dec 7, 2014
    #9
    I ended up giving up after a year of trying to merge the partitions through sanctioned ways and instead:
    • made sufficient room in the big APFS container B
    • used Carbon Copy Cloner to clone my boot volume (in APFS container A) to a fresh volume (in APFS container B)
    • booted to that to and used it for a while to see if it was a working clone
    • deleted the original volume and its container
    • resized the other container
    Boom, much more disk space.

    It's a bit disappointing that Apple's user interfaces — the GUI and the diskutil CLI — are still failing at this task.
     

Share This Page

8 March 12, 2018