Robust online/cloud backup? [MERGED]

[Boyd01]

Watching the activity in the BackBlaze window, it splits big files up into small parts before sending them. With some of the very large files I work with (up to 300gb, maybe more) there could be tens of thousands of parts. I assumed that the cpu usage was related to that (maybe also encryption?) and it will just grab as much RAM as it can.

This is what I'm backing up on my primary Mac, like I said, my use is probably unusual

Also have BackBlaze on a media server and file server that each have several terabytes of data. The unlimited storage aspect is attractive for me, will add another 4 to 8 tb to my primary Mac in the coming year but BackBlaze will still cost the same.

 

[Allen_Wentz]

Buy a fire proof safe. Cheaper than you think.

Or just toss a portable SSD in the car or elsewhere off site; even cheaper and more secure. Unless of course data is super-sensitive for some reason, in which case the security is beyond the MR fora discussion.

 

[1291030]

rclone is amazing. It has providers for a lot of cloud storage providers, and being a command line tool, easy to automate.

I'd suggest to put a bit of thought into your backup strategy and use the right thing for the right job. I got a mix of MacOS and Linux machines in my home network, and I personally differentiate between the following backup requiremnents

• Incremental backups to give me history of changes
  • Git (both linux and Mac , for code and papers written in latex)
  • Timemachine on SMB (macs, easy way of keeping a history of files I worked on in general (sans git repos))
  • linux machines , btrfs and snapper
• critical file backup
  • for things I simply can't afford to loose, I backup to an external location.
    • Code , markdown and text files are easy, I just use git.
    • for everything else, I use rclone syncing critical files to a mix of dropbox and azure blobs

 

[VitoBotta]

I have a couple of local backups — a main one for Time Machine, plus another external drive for videos.

I also want an online backup in case I lose the physical ones in my house (theft, fire etc). Ideally, this backup would be able to backup my external drive too, although I could manually sync the files if needed.

What’s the best route here? I’m happy to spend a bit, but don’t want an expensive subscription. I have Dropbox and iCloud, but assume there are better options purpose built for this.

Arq, Backblaze, and iDrive are pretty simple to use, but I've run into issues with either their reliability or performance with all three of them.

In the end, I went with Restic combined with Storj along with iDrive E2 for storage. Even though Restic is a command-line tool, it’s actually really easy to work with, super fast, and highly reliable—it's battle-tested too!

 

[m..]

Those are not really considered backups, but more like sync services. I would not rely on a sync service like that to backup my cherished photos.

I'm still using Arq with Backblaze B2 servers like I described in that thread @chown33 linked.

I have 100GB stored there and my B2 bill this month was 54 cents.

How did you get that price?
Looking at their website B2 starts at $6 per month for 1TB which still good if you need a TB.
Is yours an older contract or do they just not mention you can store less?

Ah ok I found this page on api usage https://www.backblaze.com/cloud-storage/transaction-pricing which says $0.006/GB/month

 

[splifingate]

How did you get that price?
Looking at their website B2 starts at $6 per month for 1TB which still good if you need a TB.
Is yours an older contract or do they just not mention you can store less?

Ah ok I found this page on api usage https://www.backblaze.com/cloud-storage/transaction-pricing which says $0.006/GB/month

I use a similarly-modest amount of B2 (shipped via Arc) also, and my monthly billing is something like USD .82

Just the gotta-saves . . . no need to dupe everything.

 

[blw777]

I'm a professional photographer, so I have 16TB of live data. These days I add about 1TB per year. I have two full Time Machine copies here, so I have ~60TB of disks before accounting for RAID. I have a copy on BackBlaze personal - $9/mo, and retain for a year. It took the best part of a month to get the initial BackBlaze copy up to their servers, but now it just sits in the background. It is quite reliable - I test it monthly. Nothing worse than a backup that doesn't exist or is inaccessible.

 

[gilby101]

I hear you, but what are the odds that iCloud loses my stuff at the same time that my physical backup is also lost? I think that’s a small level of risk that I can live with.

The likelihood of Apple losing your data is low. But that is not the main risk with cloud sync products. The greater risk is that you delete a file on your Mac and that deletion is synchronised to iCloud (and any other devices you have). A true backup will keep deleted files and old versions of files in case you need to recover them. A backup is also more resistant to malware and ransomeware.

I have Dropbox and iCloud, but assume there are better options purpose built for this.

In spite of asking, you seem determined that iCloud and/orDropbox are your best way forward. We are just showing some better options.

Ps. I use Arq for cloud backup.

Also what is the benefit of Arq? I see Backblaze offers personal backups for $9 a month, so it just a cost factor or is there a benefit to using Arq?

Arq is more configurable, but harder to set up. Arq maintains macOS specific file metadata, Backblaze discards it. Arq is designed for Mac and adapted for Windows, Backblaze is designed for Windows and adapted for Mac. Arq's weakness is a single developer.

Network performance is important. Backblaze (either the Personal Backup or Arq plus Backblaze B2) is slow for me in Australia. Arq plus Google Drive is much faster. You need to test before deciding.

Cost depends on storage requirements as well as the products. You have to do your own calculations.

 

[gilby101]

Also, if it backs up external drives, can I put files on an external SSD, delete them from my computer to free up storage, and not worry about it?

NO. Backup is not archive. Backup is to recover the current file system or a relatively recent one. Backblaze has a 1 year option, Arq can keep for longer. But there are no strong guarantees of long term integrity.

In you want to free up your local storage you want something like Amazon Glacier which is intended for reliable, long term storage and infrequent downloads. But you will only have the single copy (an archive) without any backup.

 

[throAU]

Those are not really considered backups, but more like sync services. I would not rely on a sync service like that to backup my cherished photos.

Whilst they are not backups in and of themselves, if you have a backup of your own on one or more disks, they are a backup to the backup in case of a house fire, etc.

Presumably if you sync everywhere AND have a disk based backup you have

• a copy of the stuff in iCloud (or whatever sync service)
• a copy on disk in your regular backup in case of being locked out of iCloud or loss of device(s)
• a copy on every device that you have the content synchronized to (in case of loss of the primary backup)

For most people that's probably enough, and certainly more than most people have (so many people have literally no backup at all).

The OP's use case (already has multiple disk based time machine backups), I'd say iCloud as a tertiary method of recovery is fine. Its pretty much what I do; quick device reinstall/recovery and working from a spare device via iCloud, DR from Time Machine backup (I have one of these off-site at work, and one of them backups my Mac desktop at home).

That said, I've considered back blaze backup for other things, I'm not 100% exclusive to Mac but all my "serious life stuff" is.

 

[throAU]

The likelihood of Apple losing your data is low. But that is not the main risk with cloud sync products. The greater risk is that you delete a file on your Mac and that deletion is synchronised to iCloud (and any other devices you have). A true backup will keep deleted files and old versions of files in case you need to recover them. A backup is also more resistant to malware and ransomeware.

iCloud keeps deleted files for 30 days.

 

[throAU]

Or just toss a portable SSD in the car or elsewhere off site; even cheaper and more secure. Unless of course data is super-sensitive for some reason, in which case the security is beyond the MR fora discussion.

Encrypted disks are a thing; if you encrypt with FileVault the disk is as good as useless to someone trying to get your data off it.

 

[Webcat86]

In spite of asking, you seem determined that iCloud and/orDropbox are your best way forward. We are just showing some better options.

Yes I’m strongly leaning to Dropbox but to be clear, this is a separate true backup option, and not the regular cloud sync. This backs up the entire computer and external drives, with 30 days or 12 months version history depending on the plan.

NO. Backup is not archive. Backup is to recover the current file system or a relatively recent one. Backblaze has a 1 year option, Arq can keep for longer. But there are no strong guarantees of long term integrity.

In you want to free up your local storage you want something like Amazon Glacier which is intended for reliable, long term storage and infrequent downloads. But you will only have the single copy (an archive) without any backup.

I’m confused. If I’ve got files on my Mac that I move to an external SSD to free up Mac space, and that SSD is backed up to an online backup, why can the files not be removed from the Mac?

 

[Pag46]

Which tier are you using, the $9/month one? I am leaning towards this for the reasons you just suggested.

Also, if it backs up external drives, can I put files on an external SSD, delete them from my computer to free up storage, and not worry about it?

I paid 178$ for two years, so about 7.4$/month, I think it was a discount for a longer contract.

About the second point, yes, as long as you tell Backblaze to backup the external drive as well and you keep it connected to the Mac.

 

[gilby101]

If I’ve got files on my Mac that I move to an external SSD to free up Mac space, and that SSD is backed up to an online backup, why can the files not be removed from the Mac?

We may be thinking differently about "removed from Mac". If it means removed from internal disk but kept on external - then no problem. If it means removed from both internal and external, then 1) you need to read carefully what the backup provider says about keeping old data, and 2) consider whether you need a backup of the cloud only files.

 

[Webcat86]

We may be thinking differently about "removed from Mac". If it means removed from internal disk but kept on external - then no problem. If it means removed from both internal and external, then 1) you need to read carefully what the backup provider says about keeping old data, and 2) consider whether you need a backup of the cloud only files.

Ah sorry if I wasn't clear — I mean the former. Move files from internal SSD to external, which is then backed up to a cloud backup account. Ideally also a Time Machine backup but it's looking like I'll need to get a larger disk soon.

 

[gilby101]

Ah sorry if I wasn't clear — I mean the former. Move files from internal SSD to external, which is then backed up to a cloud backup account. Ideally also a Time Machine backup but it's looking like I'll need to get a larger disk soon.

No worries. I went down the wrong track. What you are saying is exactly what I (and many others) do. A good local backup (yes, a bigger disk!) is essential followed by a cloud (or other off-site) backup.

 

[eyoungren]

I have a couple of local backups — a main one for Time Machine, plus another external drive for videos.

I also want an online backup in case I lose the physical ones in my house (theft, fire etc). Ideally, this backup would be able to backup my external drive too, although I could manually sync the files if needed.

What’s the best route here? I’m happy to spend a bit, but don’t want an expensive subscription. I have Dropbox and iCloud, but assume there are better options purpose built for this.

I have been using Dropbox for full backups for probably about two years now. That's mainly by habit (I started with them sometime in 2009-2010), but my plan is deliberate. And it's not for just one computer, it's for many.

I use Carbon Copy Cloner to make daily backups to my NAS. These are all sparse disk images. Of course, CCC advises that you don't use those and instead use sparsebundles. But sparsebundles are hard to move around while disk images are just drag and drop.

I have a 3TB subscription with Dropbox, plus a few dollars extra for an additional 1TB. With my own 'free' space (what I had before the subscription) that totals out to 4.1TB of space.

So, CCC is a weekly backup to the Dropbox folder on one of my Mac Pro's internal drives (an 8TB drive). Again, these are sparse disk images. Since, only changes get backed up, these disk images are not uploading the entire backup every weekend when I do this - only the changed files. My PC has a different backup app.

Several other Macs also backup to the Dropbox folder on my Mac Pro (it's always left running, 24/7). And by that method those backups become a weekly offsite backup. Since these are sparse disk images, I can pull them whenever I need them to recover.

I had the necessity to do that a few month ago. My Mac Pro is running Sonoma via OCLP (it's a 2009 model). I simply had CCC restore my backup from the disk image to a new SSD (formatted APFS). I then booted from my Sonoma (OCLP) install stick (USB) and reinstalled Sonoma. That created the recovery partition, etc. With that everything was back to normal on a new SSD.

Now, I pay my home ISP for a Gigabit connection and unlimited data. I also work from home so that and my backups justify that if anyone cares (I don't, I don't need justification for it). But since these disk images are all up on Dropbox already (via the folder on my HD on my Mac Pro), backups never take that long to update to Dropbox.

Dropbox costs me around $28 a month. That's worth it for me because again, I have several Macs and a PC that backup weekly to Dropbox.



NOTE, the ONE sparsebundle in here is because CCC wouldn't let me force a sparse disk image on that Mac for some reason. In the website screenshot you can see that it's a folder. That's why sparebundles are such a PITA to move around.

 

[splifingate]

Restic combined with Storj

Thank you for sharing these!

 

[MikeDr206]

Arq, Backblaze, and iDrive are pretty simple to use, but I've run into issues with either their reliability or performance with all three of them.

In the end, I went with Restic combined with Storj along with iDrive E2 for storage. Even though Restic is a command-line tool, it’s actually really easy to work with, super fast, and highly reliable—it's battle-tested too!

(Bump) What sorts of issues have you run into with Backblaze?

 

[ChrisA]

Thanks. That’s interesting.
I am tempted to just upgrade my Dropbox or iCloud - it wouldn’t be a full computer mirror but would let me have an online backup of all my important files and 2TB is only £8-£10 a month

Absolutely do NOT do that if you want a backup.

Here is the scenario you don't want to happen. You are working on your new novel and have 13 chapters written. Then you copy it to iCloud. All is fine.

Then unknown to you the file is corrupted and most of chapter 3 is gone. You finish up chapter 14 and then save the file to iCloud. Chapter 3 is not gone forever because you just overwrote your backup copy.

Any decent backup system will keep a version history so you can go back in time and recover from errors. This is the reason Apple's built-in backup system is called "Time Machine". It keeps a history of your changes

Yes, iCloud only costs a few $ per month for 2TB but Backblaze does keep history, has UNLIMITED storage, and is cheaper than iCloud. Oh, and Backblaze keeps a continuous backup recording changes in real-time as you make them.

Most data loss is user errors like dragging the wrong folder to trash and don't notice it until a week later.

 

[zevrix]

Yes, iCloud only costs a few $ per month for 2TB but Backblaze does keep history, has UNLIMITED storage, and is cheaper than iCloud. Oh, and Backblaze keeps a continuous backup recording changes in real-time as you make them.

Most data loss is user errors like dragging the wrong folder to trash and don't notice it until a week later.

In addition to your comment, with which I fully agree: as far as I understand, Google Drive, OneDrive (and possibly Dropbox) also keep file version history (that's what I hear from users; I myself don't use them).

 

[Boyd01]

Well, I may have finally reached the end of the road with BackBlaze. Have been very happy with them until the past few months when it started using huge amounts of RAM and CPU for extended periods while never finishing the backup of my primary Mac. Then yesterday I got this alert. To add insult to injury, this alert kept popping up endlessly every 5 minutes or so after dismissing it. Killing the backblaze processes did not stop that - doesn't appear to be any way to stop it short of uninstalling backblaze.

Apparently, this is the problem. Seems like questionable software design to load this into RAM.

"When Backblaze Computer Backup backs up your files, it loads the bzfileids.dat file into RAM to run a checksum before uploading your files to the server. If you back up an extremely large number of files, or if your computer has used the Backblaze Computer Backup software for a very long time, the bzfileids.dat file may become too large to be fully loaded into RAM, prompting an error message."

Before trying to contact support, I went through the procedure they described at the link above. That made the error messages stop but didn't seem to have any effect on the root problem. My computer still just thrashes, using huge amounts of RAM and CPU while not backing anything up. I de-selected all my external disks so that only the internal SSD is now selected for backup. That didn't help either.

Tried to contact support and it wants me to login, but the same password which works for my account doesn't work on their help site. Online chat support not an option as they are closed on the weekends. Finally just killed all the backblaze processes again and got it out of my way for now, will try contacting them Monday and see what they say. But really not impressed with their software or support at the moment, guess I need to evaluate other options.

 

[CoastalOR]

Tried to contact support and it wants me to login, but the same password which works for my account doesn't work on their help site. Online chat support not an option as they are closed on the weekends. Finally just killed all the backblaze processes again and got it out of my way for now, will try contacting them Monday and see what they say. But really not impressed with their software or support at the moment, guess I need to evaluate other options.

I can't help you with your Backblaze resource problem, but i can help with contacting support.
I also had a problem contacting support and the link sent me to Zendesk (their support provider) and my Backblaze web account password did not work. I ended up emailing support to the email address on the Backblaze contact page under the Support section:
helpme@backblaze.com

I got an email back with a support number and later a email from a support person. We eventually solved the problem, but I also found out that Zendesk does NOT use the Backblaze account password a Zendesk password. How you setup the password to Zendesk I couldn't tell you. Seems like a poor and confusing system to me.

 

[sim667]

I use Carbon Copy Cloner to backup to a NAS and then the NAS automatically uploads to Mega.

Time Machine sadly doesn't play nice with my NAS.