Eventually, I also fell victim to TimeMachine’s reliability issues

[MacBH928]

It runs in background without docker on your Mac.

this looks like an awesome solution. Its foss, its Multiplatform, it backs up to any cloud, it encrypts and looks like its easy enough to setup. And unlike cryptomator, does not require me to install mac-fuse to mess my macos system.

dd creates a byte level "copy" so its great for cloning media (HD's,etc) since it preserves the disks state byte by byte. There are plenty of free command line tools that can replace backup/duplication apps (rsync, rdiff, ditto....)

interesting, so it can do bootable copies ?

dd is most often run from the command line, in Terminal. It can copy anything, but I believe the suggestion is that @poi ran is copying a full volume to a destination volume, since they mentioned "backup OS" in their post.

And as you said, if used that way, every backup writes the full volume to the destination every time it is run. I don't know much about how that would impact longevity of the target if one were to back up frequently. Of course if the backup is interrupted or fails in some way, you've completely lost your last backup to that destination and have nothing.

dd is a particularly poor choice if done incorrectly. Just copying from a volume where active processes might be writing will result in a backup that probably is internally inconsistent or even unreadable. Backup tools like Carbon Copy and Time Machine take a volume snapshot and back up that snapshot.

If one were to use dd to backup a volume that's in use, then a snapshot should be taken, the snapshot mounted, and that volume be selected as the source of the backup. Carbon Copy and Time Machine can make snapshots on demand.

rsync is a more efficient choice since only changes are written to the destination. Also, a failure or interruption does not corrupt the entire destination volume. rsync has the same issue with regard to needing a snapshot, but the severity of that depends on the scope of what you're trying to copy. For example, backing up ~/Library would be vulnerable to changes during the copy.

Using Finder to make backups maybe also has the problem that files could be written to during the copy. I found

File exclusive access on macOS | Apple Developer Forums

developer.apple.com

You are right about files that are getting updated live. You say they create a "snapshot" , how does that work? do they freeze the current hard drive directories + files , create a copy, then backup from that? that will easily need double size the storage. I am sure there is a smarter way to do this. CCC developer said you can even continue to use the machine while you do the backup no problem.

---

I am looking for options and its overwhelming. I already found rsync, rdiff-backup, borg, rsnapshot each with its own different functions, hieroglyphics commands and perplexing lingo like "initialize a repository" AHHHHHHHHHHHHHHHH. Why can't they do it simple like SuperDuper : Copy "X" To "Y" , click "Copy Now" . Further more, why not create a GUI?

I do not want to sound entitled, but I am willing to pay for a reliable working application even if its FOSS. Just make my life easier not harder.

 

[agregson]

Old school guy running TM and rsync backups of folders.
Had always considered CCC but wondered if it’s just easier than rsync?

Cloud backup I’ve been against for all the reasons above, mostly encryption. Played with Cryptomator but not seen Arq until now. Not sure of benefits vs drawbacks between but I just know encryption means decryption for data restore so I would not want to solely rely on it; just like TM.

TM and rsync here also - older M2 mini with local drive is my backup computer and host. Local external 2TB SSD and 4TB HDD. TM to SSD, rsync to SSD and HDD from Documents and Photos via Photos backup Anywhere then rsync HDD to remote HDD. Also using Arq as additional backup and for very clean interface to browse backups and restore files. Arq also backups to Arq and Google Cloud. Backup mini also has advantage of a place to keep copies of all iCloud data downloaded for backup and a very reliable file and TM share for my primary M4 mini and MBA.

I had an issue with a memory leak on Arq and stopped using for a while but it appears to have been resolved so back in the mix.

 

[svenmany]

You say they create a "snapshot" , how does that work? do they freeze the current hard drive directories + files , create a copy, then backup from that? that will easily need double size the storage. I am sure there is a smarter way to do this. CCC developer said you can even continue to use the machine while you do the backup no problem.

A snapshot doesn't create a copy. After the snapshot is done, files that are subsequently written to are copied. I enjoyed this:

APFS: Snapshots – MacMegasite

macmegasite.com

It mentions that snapshot creation takes .01 seconds.

Carbon Copy and Time Machine allow you to use the machine while the backup is in progress because they are backing up the snapshot which is not being modified by your activities. CCC deletes its snapshot once the backup is done. Time Machine does not do that right away, but eventually prunes them; I seem to have 24 hours of snapshots at the moment. You can see all your snapshots in Disk Utility.

 

[Bregalad]

I'm impressed by how thorough you all are about your backups. I dare say that most users have no backup system whatsoever.

Oh they may use iCloud, but that's a synchronization tool, NOT a backup system. If a file is altered/corrupted or accidentally deleted on any device that change is propagated to iCloud and then to every other device. So even though a file might exist in 4 different places, it's really just one copy that's 4x as vulnerable as it would be if it was being manually copied to other devices and/or cloud accounts.

Years ago I was completely paranoid: manual copies on internal and external drives plus TM and CCC backups on external drives rotated between home, work and a trusted family member.

Then I got lazy and was down to just a single TM hard drive that died last year.

I'm finally getting serious about protecting my stuff again and this thread has helped me choose backup options going forward.

 

[bsmith736464]

Time Machine's NAS backup path has gotten much less reliable since Tahoe changed its SMB defaults. The silent failure mode is the real danger: it stops backing up with no notification, and you find out only when you need a restore.The safest move right now is to treat Time Machine as one layer, not the only one. A local rotating drive gives you a fast bootable copy. A second cloud or offsite destination covers hardware loss. If you're on Tahoe and backing up to a NAS, check Console for any "NAConnectToServerSync failed with error: 80" entries. That's the credential error introduced in 26.4 that silently kills network backups without showing any UI warning.Restic is a solid call for a second target. Fast, deduplicated, and it tells you clearly when something goes wrong.

 

[theorist9]

I mentioned this on page two, but it bears repeating:

Time Machine is a versioning system, and should NOT be relied upon as backup system. Part of the problem is that Apple promotes it as a backup system (e.g., https://support.apple.com/en-us/104984), which I believe does its customers a disservice.

It works beautifully as a versioning system, but that creates complexity, and its complexity kills its robustness.

If you want a robust backup system, consider something like Carbon Copy Cloner.

 

[WebHead]

^ I made exactly that mistake. After 15 years of hourly backups, it failed the one time I actually needed it.

(It was partially my fault, I knew my drive was failing but expected TM to throw up errors if it couldn't read it. It didn't.)

 

[BP9906]

Yeah I rely on multiple drives of file copies for this reason. Cloud I avoid but have considered a service with e2e encryption but the cost of offsite seems high for a less common scenario.

 

[theorist9]

Yeah I rely on multiple drives of file copies for this reason. Cloud I avoid but have considered a service with e2e encryption but the cost of offsite seems high for a less common scenario.

I use a safety deposit box for my off-site backups. I have a pair of drives for this, and when I want to update the offsite drive, I simply swap it out.

 

[svenmany]

I mentioned this on page two, but it bears repeating:

Time Machine is a versioning system, and should NOT be relied upon as backup system. Part of the problem is that Apple promotes it as a backup system (e.g., https://support.apple.com/en-us/104984), which I believe does its customers a disservice.

It works beautifully as a versioning system, but that creates complexity, and its complexity kills its robustness.

If you want a robust backup system, consider something like Carbon Copy Cloner.

I do agree that Time Machine is not reliable whereas Carbon Copy Cloner is. But, I don't understand why you consider one to be a backup system and the other to be a versioning one. In the current world of APFS to APFS, Time Machine works almost the same way as Carbon Copy Cloner.

- Start a backup
- Create a snapshot of the source drive
- Copy changes on the source snapshot to the backup disk
- Create a snapshot of the backup disk

There are some differences, but this is the essential flow.

With both tools, after the backup finishes, you have a complete copy of your source drive on the target drive. Finder will not show that to you for Time Machine; it only shows you the snapshots on the volume. But you can see the full file system using Terminal and everything is there.

Carbon Copy Cloner does offer you alternatives for maintaining history on the target drive, but I just use snapshots. That seems to be what they recommend now.

Are you not aware that the Time Machine volume has a full backup of your source drive? It's easy to miss it since Finder hides it from you. You can get to it using Cmd-shift-G.

 

[theorist9]

I do agree that Time Machine is not reliable whereas Carbon Copy Cloner is. But, I don't understand why you consider one to be a backup system and the other to be a versioning one. In the current world of APFS to APFS, Time Machine works almost the same way as Carbon Copy Cloner.

- Start a backup
- Create a snapshot of the source drive
- Copy changes on the source snapshot to the backup disk
- Create a snapshot of the backup disk

There are some differences, but this is the essential flow.

With both tools, after the backup finishes, you have a complete copy of your source drive on the target drive. Finder will not show that to you for Time Machine; it only shows you the snapshots on the volume. But you can see the full file system using Terminal and everything is there.

Carbon Copy Cloner does offer you alternatives for maintaining history on the target drive, but I just use snapshots. That seems to be what they recommend now.

Are you not aware that the Time Machine volume has a full backup of your source drive? It's easy to miss it since Finder hides it from you. You can get to it using Cmd-shift-G.

You're completely missing the point. Yes, Time Machine does make complete backups. But the whole point of a backup is to have something that you can rely upon if your main disk is corrupted.

Thus, in order to properly be considered an adequate backup system, it's not sufficient that something simply be able to create backups. It's also necessary that the backups be robust, and Time Machine backups are demonstrably not robust.

That's why I was very careful in my wording. Note that I never said: "Time Machine does not create backups", since I'm fully aware that it does. Instead, I said: " Timemachine should NOT be relied upon as backup system."

Are you not aware that the Time Machine volume has a full backup of your source drive? It's easy to miss it since Finder hides it from you. You can get to it using Cmd-shift-G.

That's kind of obnoxious language, and I'm not the one lacking awareness here. You are. Time Machine can't be properly considered a proper backup solution because it doesn't meet the full criteria for a backup solution, which includes robustness. That's why it can only properly be considered a versioning system.

Yes, it would be nice to also have robustness in your versioning system, but that's not its essential requirement the way it is with a backup system.

 

[svenmany]

You're completely missing the point. Yes, Time Machine does make complete backups. But the whole point of a backup is to have something that you can rely upon if your main disk is corrupted.

Thus, in order to properly be considered an adequate backup system, it's not sufficient that something simply be able to create backups. It's also necessary that the backups be robust, and Time Machine backups are demonstrably not robust.

That's why I was very careful in my wording. Note that I never said: "Time Machine does not create backups", since I'm fully aware that it does. Instead, I said: " Timemachine should NOT be relied upon as backup system."


That's kind of obnoxious language, and I'm not the one lacking awareness here. You are. Time Machine can't be properly considered a proper backup solution because it doesn't meet the full criteria for a backup solution, which includes robustness. That's why it can only properly be considered a versioning system.

Yes, it would be nice to also have robustness in your versioning system, but that's not its essential requirement the way it is with a backup system.

Then I misunderstood you. I heard you to say that TM was not a backup solution. But I now understand you meant to say it is an inadequate backup solution. I agree with this position.

I genuinely wanted to understand your point. You reacted quite negatively.

 

[theorist9]

Then I misunderstood you. I heard you to say that TM was not a backup solution. But I now understand you meant to say it is an inadequate backup solution. I agree with this position.

I genuinely wanted to understand your point. You reacted quite negatively.

Nope, please don't misrepresent what I wrote. In the post to which you replied, I did not say one thing and mean to say something else ("you meant to say...").

Quoting myself again, I said: "Timemachine should NOT be relied upon as backup system."

That's not saying it doesn't do backups. That's not even saying it's not a backup solution. That is saying you can't rely upon it as a backup solution. And I went on to explain why you can't rely upon it: "its complexity kills its robustness."

I said exactly what I meant to say. Your reponse should have been simply: "Sorry, upon re-reading your post, I realized you were clear in what you said about Time Machine, and I got it wrong. And I was wrong to question whether you understood how Time Machine works. If I had read more carefully, I would have realized you understand perfectly well how it works."

I don't appreciate your implication that this misunderstanding was because I wasn't clear in communicating my thoughts on this. Sometimes I do mess up and I'm not clear. But this wasn't one of those cases.

 

[svenmany]

Nope, please don't misrepresent what I wrote. In the post to which you replied, I did not say one thing and mean to say something else ("you meant to say...").

Quoting myself again, I said: "Timemachine should NOT be relied upon as backup system."

That's not saying it doesn't do backups. That's not even saying it's not a backup solution. That is saying you can't rely upon it as a backup solution. And I went on to explain why you can't rely upon it: "its complexity kills its robustness."

I said exactly what I meant to say. Your reponse should have been simply: "Sorry, upon re-reading your post, I realized you were clear in what you said about Time Machine, and I got it wrong. And I was wrong to question whether you understood how Time Machine works. If I had read more carefully, I would have realized you understand perfectly well how it works."

I don't appreciate your implication that this misunderstanding was because I wasn't clear in communicating my thoughts on this. Sometimes I do mess up and I'm not clear. But this wasn't one of those cases.

Calm down

 

[theorist9]

Calm down

That's exactly what everyone who doesn't want to take responsibility for messing up says when they're called on it.