iMac Kernal Panics Copying Files

Sharpshooter7

macrumors newbie
Original poster
Mar 12, 2019
5
0
DFW
My mid-2015 iMac (loaded at the time) has this hiccup where it crashes during file copying. I'm a photographer/cinematographer and my file library is fairly large (a Petabyte), kept on four and five disk RAID Arrays. This iMac has had everything replaced by Apple trying to determine why it does this and nothing has helped. Doesn't matter if the arrays are Thunderbolt or USB, even NAS. Sometimes it will copy files for a day, sometimes for 15 minutes, but at some point it is going to crash, give me the multi language box message and reboot. Obviously this makes backing up files incredibly tedious and hit and miss at best. I'd consider moving to an iMac Pro if I thought it would be any better but given the fact Apple has no idea what's causing it, I am not real confident.
 

xgman

macrumors 601
Aug 6, 2007
4,783
608
If I were you, and if I had Apple nearby, I would get a mac mini, or imac, or imac pro with a 14 day return policy, and test your setup out and see if it goes away or still happens. Then you will know what to do. ;)
 

Sharpshooter7

macrumors newbie
Original poster
Mar 12, 2019
5
0
DFW
Thanks for your thoughts! I have a great relationship with all of the DFW Apple stores, the manager at one suggested that I do exactly that. I wonder if I should do it with a current iMac? I'm afraid if I did it with a new Pro I'd never take it back :). Thanks again!
 

SecuritySteve

macrumors 6502a
Jul 6, 2017
759
899
California
Thanks for your thoughts! I have a great relationship with all of the DFW Apple stores, the manager at one suggested that I do exactly that. I wonder if I should do it with a current iMac? I'm afraid if I did it with a new Pro I'd never take it back :). Thanks again!
Is resale an option? If you're dealing with a Petabyte of data ... a Pro is really for you.
 

Sharpshooter7

macrumors newbie
Original poster
Mar 12, 2019
5
0
DFW
I don't keep anything but programs on the iMacs though, everything is on arrays. Maybe I need to wait for the much anticipated Mac Pro that never seems to be coming.
 

SecuritySteve

macrumors 6502a
Jul 6, 2017
759
899
California
Even if your storage is off-machine, you are still processing that data, and I assume running some programs against it. That's some hefty processing power. I think you'd stand to benefit from the Pro's specs. However I would also wait until this summer if you find that you can - since WWDC will likely reveal the Pro.
 

Sharpshooter7

macrumors newbie
Original poster
Mar 12, 2019
5
0
DFW
Even if your storage is off-machine, you are still processing that data, and I assume running some programs against it. That's some hefty processing power. I think you'd stand to benefit from the Pro's specs. However I would also wait until this summer if you find that you can - since WWDC will likely reveal the Pro.
Thank you for your insights Steve.
 

joema2

macrumors 68000
Sep 3, 2013
1,580
789
My mid-2015 iMac (loaded at the time) has this hiccup where it crashes during file copying. I'm a photographer/cinematographer and my file library is fairly large (a Petabyte), kept on four and five disk RAID Arrays. This iMac has had everything replaced by Apple trying to determine why it does this and nothing has helped. Doesn't matter if the arrays are Thunderbolt or USB, even NAS. Sometimes it will copy files for a day, sometimes for 15 minutes, but at some point it is going to crash...I'd consider moving to an iMac Pro if I thought it would be any better but given the fact Apple has no idea what's causing it, I am not real confident.
Are you using any 3rd-party kernel extensions? If so one of them could be crashing the OS. The procedure in this article will list them: http://osxdaily.com/2010/08/03/list-all-third-party-kernel-extensions/

How are you copying that amount of data? Finder? Have you tried copying the data using a Terminal command or via a copy utility? In general I wouldn't trust Finder 100% for copying hundreds of terabytes. It also has no "on error, resume" capability.

Are all these drives HFS+ (except for the NAS)? Are any of them exFAT or NTFS using the Paragon driver? That is a kernel extension that could theoretically crash the OS.

Are any of your RAID arrays managed by SoftRAID? That is also a kernel extension. In my experience both Paragon NTFS and SoftRAID have been reliable but an OS panic calls into question all 3rd party kernel extensions.

Before you copy hundreds of terabytes, it's a good idea to run Disk Utility First Aid on both source and destination disks. I would also run it afterward on the destination.

Copying files via Terminal: http://osxdaily.com/2010/08/03/list-all-third-party-kernel-extensions/

After copying that much data you ideally should validate it all got copied. I use Beyond Compare for this. There is a Mac version: https://www.scootersoftware.com/index.php

You can also use the 3rd party tool IntegrityChecker to create a CRC checksum and then verify that after the copy: https://diglloydtools.com

There are various other 3rd part copy utilities but I haven't tried these, just listing them. Before you use any copy utility or Terminal command full scale, it should be tested on a small data set and the files checked.

CopyCatX: https://www.subrosasoft.com/product/copycat-x/

UltraCopier: http://ultracopier.first-world.info

RapidCopy: https://itunes.apple.com/us/app/rapidcopy/id975974524?mt=12

Hedge (designed for camera offload but useful for reliably copying files): https://hedge.video
 

Sharpshooter7

macrumors newbie
Original poster
Mar 12, 2019
5
0
DFW
Are you using any 3rd-party kernel extensions? If so one of them could be crashing the OS. The procedure in this article will list them: http://osxdaily.com/2010/08/03/list-all-third-party-kernel-extensions/

How are you copying that amount of data? Finder? Have you tried copying the data using a Terminal command or via a copy utility? In general I wouldn't trust Finder 100% for copying hundreds of terabytes. It also has no "on error, resume" capability.

Are all these drives HFS+ (except for the NAS)? Are any of them exFAT or NTFS using the Paragon driver? That is a kernel extension that could theoretically crash the OS.

Are any of your RAID arrays managed by SoftRAID? That is also a kernel extension. In my experience both Paragon NTFS and SoftRAID have been reliable but an OS panic calls into question all 3rd party kernel extensions.

Before you copy hundreds of terabytes, it's a good idea to run Disk Utility First Aid on both source and destination disks. I would also run it afterward on the destination.

Copying files via Terminal: http://osxdaily.com/2010/08/03/list-all-third-party-kernel-extensions/

After copying that much data you ideally should validate it all got copied. I use Beyond Compare for this. There is a Mac version: https://www.scootersoftware.com/index.php

You can also use the 3rd party tool IntegrityChecker to create a CRC checksum and then verify that after the copy: https://diglloydtools.com

There are various other 3rd part copy utilities but I haven't tried these, just listing them. Before you use any copy utility or Terminal command full scale, it should be tested on a small data set and the files checked.

CopyCatX: https://www.subrosasoft.com/product/copycat-x/

UltraCopier: http://ultracopier.first-world.info

RapidCopy: https://itunes.apple.com/us/app/rapidcopy/id975974524?mt=12

Hedge (designed for camera offload but useful for reliably copying files): https://hedge.video
Dear IS,

Thank you for the time you spent on that response! Let me go through each of those questions and get back to you. Four of my arrays are Drobo 5D units, one is an 8 bay QNAP NAS with their 8 bay extension unit, and six of the arrays are OWC RAID enclosures (all Softraid). I have Hedge but find it extremely tedious when backing up huge amount software data like this. Typically I am not backing all of my files up but, in this instance, I am making back ups of everything I have at the moment on eternal drives and will put them in padded envelopes in a third location. This is just so I know that if the worst happens I have triple redundancy, my home, my hangar office and climate controlled storage for the master drives.

I have gone back and checked the integrity of each and every file in these folders (ugh!) to make sure that they are complete and that there's no corruption. Generally the largest folders I'm copying are a couple of TB's, most are in the .3 to .5 TB range. Here's a typical back up window:



Thank you again for your time and expertise!

JW
 

Attachments