Invalid Node Structure Error, need help backup and fix

[df925]

Hello,

*** Warning, long read, I tried to be as detailed as possible ***

2013 iMac 21.5, 1TB Fusion

The other day I restarted the computer, and it would boot about 2/3 of the way on the progress bar, and then shutdown.
I booted in verbose mode, and captured this message
Executing fsck_hfs (version hfs-407.50.6)
...
Invalid node structure
(4, 160565)
** The volume Macintosh HD could not be verified completely.
...<Emergency>: Boot task failed:...
... Shutting down in 3 seconds.

Create image of device
I then used Arch Linux boot USB on iMac and did a dd_rescue on the full device
dd_rescue /dev/disk2 mac.dmg imac.log
and got a 1 TB image file.

Put image on external USB drive
on a separate computer, I copied the image to a usb drive
dd if=mac.dmg of=/dev/disk4

Able to see Partitions and mount Apple boot
Disk mac.dmg: 931.5 GiB, 1000204886016 bytes, 1953525168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: FACF090D-3BE6-44F3-A614-9D4F4EDB538B

Device Start End Sectors Size Type
mac.dmg1 40 409639 409600 200M EFI System
mac.dmg2 409640 1952255591 1951845952 930.7G Apple Core storage
mac.dmg3 1952255592 1953525127 1269536 619.9M Apple boot

Unable to mount Apple Core storage
So I'm unable to mount the Apple Core storage in linux, bad superblock
but I am able to mount the Apple boot partition.

Able to start iMac in Single User Mode
So I was able to start in Single User Mode, and I immediately copied my Pictures folder onto an external USB with HFS
mount -uw /
mkdir /Volumes/bk
mount -t hfs /dev/disk3s3 /Volumes/bk
cp -R /Users/foobar/Pictures /Volumes/bk/photos
This took all 12 hrs, ~750GB

So now what?
I have not powered down my iMac from Single User Mode.
1. Should I shutdown and reboot into Single User Mode, and run fsck -fy to see if that corrects the invalid node structure issue? Could this make this worse, and prevent me from accessing the files?

2. Should I perform some other type of backup before I shut it down, ie cp -r /Users /Volumes/bk/Users (probably ~1TB), is there something better to use other than cp?

3. How would you guys proceed?

I got the photos which was the biggest thing. Now I have some breathing room to do some additional trial and error things without as much worry. Obviously I'd like to recover more, but I'm very cautious and thats why before I did anything I did a dd of the whole drive. It just doesnt seem like the original DD i did was good enough? but I'm not even sure how to check the validity of the original DD.

thanks for keeping with me this far, I know its a long read.

If you guys have any suggestions/comments/feedback/advice, I'm all ears, and I promise to keep you guys up to date on what I do, and post a final conclusion (I hate when threads just stop)

thx
Davis

 

[df925]

One piece of news, I'm able to reboot into single user mode, and have access to the file system (my heart stopped when I had to reboot, then when it came up with the file system, I was able to let out a huge sigh of relief )

FYI, I've planned on doing the following

1. cp -Rpv /Users /Volumes/bk/Users

After thats done, probably 12-24 hrs as it has to copy almost 1TB​

2. dd conv=noerror,sync if=/dev/disk2 of=/Volumes/bk/imac.dd

I think this will give me a raw image of my iMac HD, just another backup.​

3. see if I can mount the imac.dd in a separate Mac
4. reboot in single user mode and run fsck -fy and then restart normally

I'm just sitting around waiting for the next 12-24 hrs, so if you guys have any suggestions, they are still welcome!

thx
Davis

 

[chown33]

I'm not entirely sure, but I think if you make a disk-image (man hdiutil), then use it as the of= arg to dd, that it'll write a mountable disk-image file.

I'm sufficiently unsure of this that I probably wouldn't waste time doing it from the damaged disk. Instead, I think outputting to the ".dd" file is sufficient.

There's probably a way to compress dd's output in a pipeline, which might make it write faster, but that would depend on the write speed (or lack of it) for the device on /Volumes. If there's lots of zeroed blocks, compression will squeeze those down a lot. 'gzip' would be my first choice for a compression filter on stdout, mainly because it's widely readable.

BTW, unless you've done something with the disk mounter, it will mount in /Volumes, with an 's', not /Volume (no 's' at end).

 

[df925]

I'm not entirely sure, but I think if you make a disk-image (man hdiutil), then use it as the of= arg to dd, that it'll write a mountable disk-image file.

I'm sufficiently unsure of this that I probably wouldn't waste time doing it from the damaged disk. Instead, I think outputting to the ".dd" file is sufficient.

There's probably a way to compress dd's output in a pipeline, which might make it write faster, but that would depend on the write speed (or lack of it) for the device on /Volumes. If there's lots of zeroed blocks, compression will squeeze those down a lot. 'gzip' would be my first choice for a compression filter on stdout, mainly because it's widely readable.

BTW, unless you've done something with the disk mounter, it will mount in /Volumes, with an 's', not /Volume (no 's' at end).

The dd of /dev/disk2 is just a precaution, and just to have the raw image incase I need it, or to play with it. would rather run fsck on the dd image as opposed to the actual disk first.

regarding the mount into /Volumes, I corrected my type-o, but unfortunately in Single User Mode, usb drives do not automount.

thx
Davis

 

[chown33]

The dd of /dev/disk2 is just a precaution, and just to have the raw image incase I need it, or to play with it. would rather run fsck on the dd image as opposed to the actual disk first.

I completely agree.

 

[Fishrrman]

If you can successfully get your personal data OFF OF the internal drive -- or at least the most important data (old Mail, pics, music, etc.) -- then I'd nuke the internal fusion drive "back to zero" and start over.

Hmmm.... a 2013 has a 128gb SSD portion, is this correct?

If that's the case, I'd NOT re-create a "fusion drive".
Instead, I'd just erase both the SSD and the HDD, and "let them exist as standalone drives".
I'd set up the 128gb SSD to be the boot drive.
If the HDD is still usable, I'd keep "large libraries" on that one.

And keep BOTH drives backed up...

 

[df925]

1. cp -Rpv /Users /Volumes/bk/Users

After thats done, probably 12-24 hrs as it has to copy almost 1TB​

2. dd conv=noerror,sync if=/dev/disk2 of=/Volumes/bk/imac.dd

I think this will give me a raw image of my iMac HD, just another backup.​

3. see if I can mount the imac.dd in a separate Mac
4. reboot in single user mode and run fsck -fy and then restart normally

1. completed, second sigh of relief
2. failed, "Operation not permitted", seems to do with SIP
Recommendations?

 

[df925]

Hmmm.... a 2013 has a 128gb SSD portion, is this correct?

If that's the case, I'd NOT re-create a "fusion drive".
Instead, I'd just erase both the SSD and the HDD, and "let them exist as standalone drives".
I'd set up the 128gb SSD to be the boot drive.
If the HDD is still usable, I'd keep "large libraries" on that one.

And keep BOTH drives backed up...

I'll check on the size of the SSD. seems about right, since the HDD was 1TB, and has a 200gb partition for recovery? but I have to confirm

Interesting concept on splitting them, I'll definitely think about it.

and yes, keep everything backed up. I went crazy at costco, 3x5tb non-powered drives ($100 each), just to be sure!

 

[df925]

Just another update incase anyone is interested.

I booted single user mode, and ran /sbin/fsck -fy and still got
"The volume Macintosh HD could not be verified completely"

So then I decided to boot into Rescue Disk.

From Rescue Disk, I tried to run first aid on the disk, and got a similar issue, unfortunately I didn't document that information.

However from Rescue Disk Mode, I was able to successfully create a disk image of Macintosh HD.

so another Hurray!, I have a full .dmg of my computer.

The only thing I can think of is maybe when I restarted last, Apple tried to upgrade my HFS+ to AFS? Thoughts anyone?

anyways, for anyone that gets the Invalid Node Structure issue, I was able to get a disk image that had what I can tell was all the files from my computer using Rescue Disk Mode.

Next step will be to try and reinstall OS X on an external USB, get that upgraded Mojave, and see if the disk utility in Mojave can fix my internal drive Invalid Node Structure issue.

thx
Davis