Best practices on how to avoid (software) data loss?

[toke lahti]

Story:
For some years, I've needed to start "spring-clean" on my storages. Had bought more usb hubs and adapters and extension cords etc.
Checking disks and thinkin how to organise the archives again, I notice that a "local backup" of an off-site raid-5-nas has problems.
Digging deeper, I found out that in Feb 2018 I had used it to transfer stuff from one disk to another. I did it with old mbp and started with usb3-expresscard, which prooved to be fleaky, so I copied everything again with usb2 (the other drive was connected with fw800).
Archive that was copied was 2.7TB of tv-recordings that were not important and I had no time to verify the copy.
Now I found out that these files have been mixed with my archive of family videos!
Opening up my mom's funeral or sister's wedding opens up some random tv program!

Of course I have prepared for disasters, so I ask my dad to switch on my "off-site-nas" at his house, since I can't get it on remotely. He tells me: "It does not turn on." OMG. And I just wrote everything I had on-site to that problem disk.

In parallel reality we could use optical WORM disks, that would now be cheap and something like one or half a terabyte. I'd wish to be in that reality. In our reality, I don't think we have any other econimical option to archive our personal digital life than hdd's.

But since I'm here (in our reality), I'd like to hear experiences and ideas on how to monitor and preserve data integrity in "advanced home enviroment".
What would be the best software for this?
Or just brute force? Meaning multiple computers dedicated to making multiple clones of everything and verifying that files that were not touched haven't been corrupted.
(My biggest problem has always been that I need to do something else with that computer doing the copying. Which always takes too many hours. In this case changing from usb3 to usb2 extended the copy time from 5 hours to 50 hours. Then there would be additional 50 hours for verify...)

How often this (copying stuff corrupts other stuff in the same disk) can happen?
I understans some of filesystem basics. This problem could happen just because of corrupted file allocation table where data is pointed to be in a wrong place. And of course, hpf+ does not verify the copying process. Is there a terminal command that does verify? Ditto?

 

[Fishrrman]

I'd STAY AWAY from RAIDS and/or NAS for home usage.

Get one or more backup drives.
They can be platter-based or SSDs.

Get a good backup app like CarbonCopyCloner or SuperDuper and use it frequently.

That should do it.

 

[Boyd01]

I don't know what is "best", but here's what I do. I have a 2012 quad Mini Server on my gigabit ethernet LAN with one 5tb external disk for legacy archival storage (stuff I rarely need but don't want to lose) and another 5tb disk for Time Machine backups of my primary Mac. There are two additional 5tb disks attached that are mounted automatically late at night, cloned with Carbon Copy and then dismounted.

I also have a 2tb Samsung T7 bootable clone of the internal 2tb internal SSD on my primary computer and manually update this frequently, storing in another part of the house when not in use. I also have BackBlaze doing continuous cloud backups of the primary computer and the server. I run Windows software under Parallels on my primary Mac, so that is all backed up with everything else.

I have a 2014 Mini as a media server, located inside a cabinet in another room. All the media is on an external 4tb SSD and is automatically cloned to an external hard drive that is mounted/dismounted late every night. There is also a second external backup drive that I manually rotate periodically and the unused one is stored in another room. The media server is also continuously backed up to the cloud with BackBlaze.

In addition to all this, I have multiple old hard drives with clones of old computers and old versions of MacOS. Regarding corrupted files, I used a 2tb Time Capsule for a number of years, and there were at least three times when I got an alert that the backup failed an integrity check and needed to be deleted and started again from scratch. That did not inspire much confidence. I stopped using the Time Capsule for backup, although it still provides wifi in my home. My primary Mac, media server, file server and two Apple TV's are all hard-wired on gigabit ethernet.

When I was making some big changes to my setup recently, a very old Windows USB disk had lots of errors when I tried copying it to a new disk. There was nothing especially important there, but I was able to salvage about 80% of it by cloning with Carbon Copy (it literally took a week!) and I also pulled a number of files from the BackBlaze backup.

I copied several other legacy Macintosh hard drives to a single 5tb disk and there were no errors at all. For me, the key is to have multiple backups in different places. And I also think it's important to have some of these offline, where they are secure. If you are ever hit by a ransomware attack, any disks that are mounted are at risk, and things like BackBlaze or Time Machine may just copy the compromised files.

 

[toke lahti]

I don't know what is "best", but here's what I do. I have a 2012 quad Mini Server on my gigabit ethernet LAN with one 5tb external disk for legacy archival storage (stuff I rarely need but don't want to lose) and another 5tb disk for Time Machine backups of my primary Mac. There are two additional 5tb disks attached that are mounted automatically late at night, cloned with Carbon Copy and then dismounted.

I also have a 2tb Samsung T7 bootable clone of the internal 2tb internal SSD on my primary computer and manually update this frequently, storing in another part of the house when not in use. I also have BackBlaze doing continuous cloud backups of the primary computer and the server. I run Windows software under Parallels on my primary Mac, so that is all backed up with everything else.

I have a 2014 Mini as a media server, located inside a cabinet in another room. All the media is on an external 4tb SSD and is automatically cloned to an external hard drive that is mounted/dismounted late every night. There is also a second external backup drive that I manually rotate periodically and the unused one is stored in another room. The media server is also continuously backed up to the cloud with BackBlaze.

In addition to all this, I have multiple old hard drives with clones of old computers and old versions of MacOS. Regarding corrupted files, I used a 2tb Time Capsule for a number of years, and there were at least three times when I got an alert that the backup failed an integrity check and needed to be deleted and started again from scratch. That did not inspire much confidence. I stopped using the Time Capsule for backup, although it still provides wifi in my home. My primary Mac, media server, file server and two Apple TV's are all hard-wired on gigabit ethernet.

When I was making some big changes to my setup recently, a very old Windows USB disk had lots of errors when I tried copying it to a new disk. There was nothing especially important there, but I was able to salvage about 80% of it by cloning with Carbon Copy (it literally took a week!) and I also pulled a number of files from the BackBlaze backup.

I copied several other legacy Macintosh hard drives to a single 5tb disk and there were no errors at all. For me, the key is to have multiple backups in different places. And I also think it's important to have some of these offline, where they are secure. If you are ever hit by a ransomware attack, any disks that are mounted are at risk, and things like BackBlaze or Time Machine may just copy the compromised files.

I also have CCC to use. Recently I noticed that you can't clone or block copy a disk that can't mount. That's pretty bad limitation for recovery, but like they support says, CCC is not for recovery.

How (with what sw) you do your "automatically cloned to an external hard drive that is mounted/dismounted late every night" thing?

 

[toke lahti]

I'd STAY AWAY from RAIDS and/or NAS for home usage.

Get one or more backup drives.
They can be platter-based or SSDs.

Get a good backup app like CarbonCopyCloner or SuperDuper and use it frequently.

That should do it.

You might be right.
Using NAS was to cut hdd costs.
I had an idea that because you need 3 copies of data to preserve it and data is lost because of technical failure or because of human error, with raid-nas, you need only 2 copies, because raid redundancy handles the technical side.

When my hdd's were filled with low profit professional video stuff, cutting costs was necessary.
Now it's different with also still lightly decreasing price of storage.

My thoughts are, that too large archive hdd gives you an idea that you can use it also to something else and that where I went wrong. If I had had 2 smaller hdd's with one hvaing my archive and other used in transferring copies, the fleakiness of copying wouldn't have destroyed my archive. Damn sales prices in amazon...

 

[HDFan]

I'd STAY AWAY from RAIDS and/or NAS for home usage.

Why? Although not bootable, don't see why it couldn't be one of the 3 backups in a 3-2-1 strategy.

 

[Boyd01]

How (with what sw) you do your "automatically cloned to an external hard drive that is mounted/dismounted late every night" thing?

Very simple, I use Carbon Copy Cloner. You can set a schedule for your backups and there are various options that include mounting/dismounting the disk.

 

[hobowankenobi]

There are many angles to cover:

hardware failure
volume failure
bit rot
directory structure issues
file system issues
user mistakes (overwriting files, deleting wrong files, etc)

APFS is better than HFS to help prevent much of the above..but other file systems are even more robust. ZFS and BTRFS are worth considering for their robustness and self-healing...as a better file system for long term data security. File system snapshots are another layer of protection, especially from versioning or user errors. Nothing to do or rely on... the way we do OS or file-level versioned backups. And, most importantly, without needed many times the space to store multiple versions of all files.

Many of the risks that are factored in and addressed at enterprise scale feel very, very low risk to end-users...to the point many are unaware or unconcerned about the dangers at all.

For me to feel safe, I want archives to be housed:

On multiple volumes
On at least one RAID or another redundant device (see ZFS pools, etc.)
Automatically verified
Easily manually verified
Easily restored

That takes care of the most likely causes of data loss. On top of that, we need to consider disaster recovery: fire, flood, theft, alien attacks, etc.). For this, the best answer is off-site. Today, that typically means cloud storage. But it could mean rotating external drives that get stored off-site, or, a NAS online at a friend or relatives house (private cloud).

So...

CCC (or similar software) to rotating external drives is good enough for most folks, most of the time. We could throw out a number like...it is 95% safe (covering 95% of risks of data loss).

But it is difficult to remove the last few percentage points of risks of data loss...without substantial changes.

For me, the compromise means backups to local drives on occasion via CCC or similar, as well as an offsite Synology NAS that has redundant drives, data snapshots, self-healing, and automated daily volume verification built in.

While Synology BTRFS is not as robust as other file systems such as ZFS, it is for me the sweet spot between cost/ease of use/relibilty, and should be more robust than any workstation filesystem, plus other benenfits.

We all have to decide our comfort level with risk, and what the data in question is worth.

 

[MisterSavage]

I'm a big proponent of 3-2-1.

"A 3-2-1 strategy means having at least three total copies of your data, two of which are local but on different mediums (read: devices), and at least one copy offsite."

I backup my Mac to a a Time Machine drive and have online backup via BackBlaze. Even if my place burned to the ground and I lost everything I could still get my data back.

 

[James_C]

Roughly only 34% of US Computer users backup their data at least once a month. Idealy for the majority of users the backup system should be as frictionless as possible, the more manual steps involved the less likely you will stick to a regular backup system. Of course if you are the type of person that is organised to connect an external drive, start a backup then move it offsite then great. You have to be honest with yourself - are you going to remember to do any manual steps as regular as you need to.

The other part is how important your data is, if you lost it. I always use the mindset that it is not the case of if your drive will fail, but when it will fail. On many Macs these days it is not always possible to easily remove the SSD from the computer. So if the Mac dies you may have difficulty recovering your data. The more important your data is, then the more backup systems you should invest in.

So if you are the type of person who does not want to bother with offsite backups or configure backup software, a sensible start point is Time Machine which is painless to set up and will backup in the background, which is part of Mac OS. All you need is an external drive, can be a cheap HDD or SSD. If your Mac is a desktop then this can be permanently connected to your Mac. If your Mac is portable, I recommend that you invest in a Time Machine compatible router and connect it to that. Your Backup Drive should be 2x the size of the drive that you are backup up.

Time Machine will do incremental backup every hour. As well as being able to do a full machine restore, Time Machine will allow you to restore files and folders from any point in the past where the Time Machine backup has started. So if you accidental delete or overwrite a file and you have a time machine backup you can recover it. This has saved my bacon a number of times. Time Machine has also allowed me to fully recover data on a failed HHD there times in the past 12 years.

However Time Machine is not prefect (I have one occurrence of Time Machine not being able to recover a failed drive ) ,can be slower to recover a whole system that something like carbon copy cloner, and does not create bootable drives like Carbon Copy Cloner can.

So if you are the type of person that wants a 'Fire and Forget' backup solution then Time machine is better than nothing, however I would also have at least a second backup system.

The second backup system ideally should allow you to create backups that can be kept offsite, as a local backup may not protect you from fire or theft. Carbon Copy Cloner would be the ideal for those how are organised enough to backup their data and keep a copy offsite. If this is not you then I suggest you look at a cloud backup solution like Backblaze. The main problem with cloud backups is the time needed to recover your data, either by downloading your data or the backup service sending you a copy on a HDD. Again getting your data back in a day or so, is better than not getting it back at all.

 

[monokakata]

As for NAS, I have a 4 drive (2 x 2 mirrored) Synology on the house network. It's really there because we have 4 or sometimes 5 Macs running that can all use shared storage. It's convenient (an obvious thing to say, I know).

I use it as a "I think I'll put another copy of this folder up there," or "I think I'll fire up Chronosync and make sure this important folder is mirrored on the NAS." I could, but don't, automate that kind of thing.

The OP said nothing about needing shared storage, I know. But, OP, if that's attractive to you, then a good NAS with mirrored drives and its own backup might be a very good choice for you.

 

[Boyd01]

I have multiple Backblaze accounts and am very happy with them. However, people need to realize its limitations. It is not a clone of you disk and doesn't save all the things that Time Machine does either. Basically, it only backs up your user files (documents you have created) and does NOT back up your system files or your software. So, if you restore from Backblaze, you will need to re-install MacOS separately and will also need to reinstall all your software from the original source. BackBlaze ignores a lot of file types (such as installers and virtual machines), although you can custom configure it to allow many of these to be backed up. But it cannot be configured to back up your System or Applications folders.

 

[toke lahti]

One thing that NAS removes from the list of failures is just the one that happened to me: fleaky connection to DAS.
I got my nas resurrected by changing the power (12y old N5200, high quality looking box from the inside, btw), so my data is safe.
Would not be fun to buy a new nas now, I use that for few days a year.

A decade ago I had something like 1tb of personal video and 5-10tb of pro video archived. Then it was not feasible option to include these to TM. Now I’m considering at least to add those personal stuff to it. Or just throw them to one of the backups of a backup of a backup...

I think I might get a new onsite-nas, if there’s an elegant and economical option available.
I’d like it to be quite silent and most importantly very low energy use. So it should have very good power management and it could be always on and not use much of juice and especially not spin the disks, when not necessary. I’m not sure if it should have 2.5” or 3.5” bays, 5-6 of them to have hot spare for raid5 online. And I’d hope that modern nas does have option to ssd cache. 2 ssd’s in raid1 might be nice, just small ones are enough (maybe 256gb each), half reserved to new files and the other half caching the latest and most used files already in raid5.
If this would be done in DIY project, could the power optimization be very hard?

 

[hvfsl]

One thing that NAS removes from the list of failures is just the one that happened to me: fleaky connection to DAS.
I got my nas resurrected by changing the power (12y old N5200, high quality looking box from the inside, btw), so my data is safe.
Would not be fun to buy a new nas now, I use that for few days a year.

A decade ago I had something like 1tb of personal video and 5-10tb of pro video archived. Then it was not feasible option to include these to TM. Now I’m considering at least to add those personal stuff to it. Or just throw them to one of the backups of a backup of a backup...

I think I might get a new onsite-nas, if there’s an elegant and economical option available.
I’d like it to be quite silent and most importantly very low energy use. So it should have very good power management and it could be always on and not use much of juice and especially not spin the disks, when not necessary. I’m not sure if it should have 2.5” or 3.5” bays, 5-6 of them to have hot spare for raid5 online. And I’d hope that modern nas does have option to ssd cache. 2 ssd’s in raid1 might be nice, just small ones are enough (maybe 256gb each), half reserved to new files and the other half caching the latest and most used files already in raid5.
If this would be done in DIY project, could the power optimization be very hard?

Qnap (and many other manufacturers) offer what you want with off the shelf solutions. Alternatively you can build up a FreeNAS box using an old server from eBay. There are 2 types of SSD caching, read and write. The write provides a buffer for writing to the HDDs and the Read cache caches the most used files. You would generally need at least 3 SSDs to take advantage of both: 2x for the write cache in RAID1 and 1x for the read cache (since this is just a copy of what is on the HDDs already).

I built one out of a low power Intel Atom motherboard and just set the HDDs to spin down after 30mins of inactivity.

 

[Fishrrman]

James gave an excellent overview on backup in #10 above.

One thing I would add:
For an "offsite" backup, one doesn't have to have another "building" in which to store the backup.

In my working days, I kept my offsite backup in my locker at work, but now that I'm retired, I just keep it in my car. I use an nvme SSD in a USB3.1 gen2 enclosure and keep that in a plastic ziploc bag with a desiccant inside. Doesn't seem to mind the heat or cold.

There's some data on one partition that I want "protected", so that's the only one of my drive volumes that is encrypted. The other stuff, I don't mind if it's left "in the clear".

This way, even if someone stole the car, all they'll get "is a drive".
And being a cloned backup, all I'll lose is "one backup", which can be easily replaced.