Time Machine backups painfully slow on NAS (smb)

[Zerex71]

Thanks to everyone who had input to my original questions. I've been a busy beaver lately with my new NAS (I did go with the EX2 Ultra) and haven't had any issues to speak of, at least none that have been anything other than the amount of time it takes to do backups. I'm still running Catalina (10.15.7), and for a while, my CONOPS included alternating backups between my AirPort Time Capsule (APTC) and my new NAS. I let that go on for a couple of weeks before I finally cut over to just using the NAS. It's not that there was anything detectable going on with the APTC that required it, I just needed to bite the bullet while it was still operational and cut the cord. After a good soak period with the NAS, I'll go on about erasing the APTC while I can still talk to it.

What I can report is that mainly, as other users reported, the initial backups took forever--for me, days--and slowed my iMac to a crawl and also crash-restarted it in the middle of the night, so it had to start all over again wherever it left off. It was aggravating but not entirely unexpected. Eventually, it completed after about, I'd say, 4-5 days (it was kind of hard to tell when something stopped successfully and a next "layer" of backup proceeded -- I just kind of had to wait until the backup increments kept decreasing in size). After about a week or so, give or take, things planed out, and backups have been going on pretty regularly since. (To address performance comments someone made above, I am running with hardware encryption for the exact reasons cited -- I'm savvy enough to know how critical it is to have protected, privatized data -- so perhaps that's contributing to the slowness.)

I haven't tried to retrieve anything from TM yet, so I can't speak to how well that works.

I'm also in the middle of a big data decluttering/de-duplication and photo organization project to go with this that I expect will go on for a while. I'm working with between 2-3 TB on my local drive, but I have multiple very large Photos libraries for starters that need to be broken apart, sorted, and offloaded. One feature of having a personal cloud is that it does have enough functionality for media on it for my purposes and now I can really see the value of trusting the device enough to offload all the photo libraries I've built up over the years. They take up a good percentage of my current storage space, and this is a consideration because when I get a new iMac, all they come nowadays is SSD, and that's super-expensive, so if I can get my local storage down, I might not have to spend the extra money. Anyway, I'm finding that it's perfectly fine to offload them and I don't really notice any issues with not having things in Photos -- I can export any way I like, save off EXIF data, and use a third-party app to adjust file attributes to my liking. (Sorry for the minor tangent there.)

I'm less worried about Big Sur now though that I've had this device for a while and seen the comments. Most of what it seems like is Synology users having issues. I see no reason that Apple would wreck a capability like this, but you know, they did remove that headphone jack from the iPhone...

 

[richsadams]

Also late to the party here but I believe I have reached the end of the line when it comes to TM. I have read every post and appreciate all of the suggestions, to get it to work, but no thanks. Whatever happened to "It just works"?

Prior to updating my iMac (Retina 5K, 27-inch, 2017) to Big Sur, there were no problems with TM backups to my 2TB Time Capsule (the latest, last model Apple made). After updating TM backups became unbearably slow and eventually would never finish. They still worked normally on my wife's iMac and my MacBook, both still on Catalina.

After following Apple's (and other's) recommendations I wiped the Time Capsule drive and reformatted it to APFS. The initial backups for all three Macs worked great and finished overnight. My hopes were high but immediately dashed since then as TM backups for all three are now unbearably slow. The first one since the full backup on my iMac says it will take 8 hours to back up a little over a gig. Since the initial update I've only opened mail and worked on a few websites. I know I have not generated more than 1 gig of new files. No idea why it's so large.

All of that said, I have no interest or the time to jump through all of the hoops suggested here to get an NAS or regularly connected drive to work with TM. It's clear in my mind that since dropping Time Capsules that Apple is no longer interested in backup support of any type. In this day and age of ransomware, malware and more that seems pretty incredible. Even to restore a failed HDD or SSD requires a good backup plan.

I've set up iDrive for all three of our Macs and iPhones for emergency purposes, but would still like a "hard copy" locally since recovering files stored in the cloud can take a very long time.

Has anyone moved to another option? I have Super Duper, but it doesn't play well with Big Sur, at least yet and I don't want to have to set up three different backup drives. I'd be willing to set up an NAS if it doesn't involve TM.

Thanks for listening, welcome any thoughts and TIA!

 

[haralds]

I use the following strategy:
- Documents, Desktop, and critical items are stored in and synced with iCloud
- TimeMachine is slow to begin with, but working ok for me backing to a Mac Mini Server that I reverted to Sierra, when Apple had the last full server support. Upgrading it to Catalina created a disaster.
- Periodic full backups with Carbon Copy Cloner. The first backup consumes the full copying time, but after that incrementals are quite fast.

 

[haralds]

The wonderful Howard Oakley has this to contribute: Death by SMB
He has articles on macOS and Macs with great technical insight.
Check out THE ECLECTIC LIGHT COMPANY

 

[richsadams]

Quick follow-up. I've left TM working and noticed that when it runs (hourly) it shows the file-size to be updated and eventually the time calculation. "Preparing backup" seems to take an inordinately long time now.

The most recent backup showed about 200MBs and that it would take 1 hour. However, I looked at it about 10 minutes later and the backup had completed.

So besides it indicating the file size updates being much larger than any I've added or changed, it also looks like TM's ability to calculate how long it will take is wildly off.

I did attempt to recover a small file I had on my desktop a few hours ago. Loading took a good 45 seconds or so, but it finally showed up and I was able to recover it.

Maybe there's still hope?

I'll keep an eye on it and report back.

In the meantime, I'd still appreciate any TM alternative recommendations.

BTW, thanks very much for the link @haralds. The blog post you referenced explains a lot:

Last Week on My Mac: Death by SMB

Slow performance when backing up to a network share is mainly down to SMB. Without its improvement, Time Machine over a network is still dead in the water.

eclecticlight.co

Update to my update: Continuing to watch TM. I still see it showing very, very large backups when very little has been modified or added. Most recently I added three photos, a couple of small text files, answered and created some basic emails. Between the last backup and that activity TM showed that the newest backup was for 1.06GBs and that it would take three hours. It was done in about 10 minutes. I'm pleased that it seems to be working, but something's not right and I still don't trust it.

The screenshot below is a typical backup now. This particular one had no file changes (by me) and indicates that it will take five hours to move less than 350MBs. There was no other activity at the time. It was completed in less than 7 minutes.

 

[gurple]

It seems to me that, in addition to the poor performance of SMB in general, a lot of this slow performance is Time Machine on the client performing copious reads across the sparsebundle bands as it performs its internal management and thinning of the backup content.

As some have pointed out, there is a lot of traffic, which can be watched with nettop or Wireshark, that vastly outsizes the minimal files-to-be-backed-up in each session.

There is so much I/O taking place over lousy protocols that it can't get anywhere near to taking advantage of the available network throughput. …slow serialism iterated over millions of files in thousands of bands.

It would be delightful if Apple would provide a first party iSCSI initiator in the OS. I suspect that it would yield better performance on the same infrastructure.

 

[RiderX]

After installing Big Sur 11.3 the following Timemachine backup was a bit over 4GB and took less than 15 minutes. My "NAS" is a crappy WD Mycloud. I connect with ethernet and use AFP to serve the encrypted sparsebundle (APFS) on the NAS. This is not terribly fast, but what you can expect normally.

The sparsebundle consists of thousands of folders and millions of small files. AFP is a simple protocol to serve them well. SMB may use more "artificial intelligence" to **** that up.

 

[tehabe]

My Time Machine didn't work at all over SMB3 but now it seems to be working again. I use an Synology NAS.

 

[richsadams]

My TM backups to our 2TB Time Capsule seem to have settled down on our three Macs. Backup file sizes have come down from the crazy 1GB to 6GB range to somewhere around what I'd expect. The time estimates are still crazy though. The "Preparing" stage can still be lengthy and it can show a 325MB backup will take six hours but ends up completing in 10 minutes.

It does appear to be working though and I've been able to recover test files without any issues, although that can take a painfully long time as well.

I can only hope that somewhere in the shiny new halls of the Apple campus that someone is still working to improve TM.

I'm using iDrive for cloud backups, but I'd still like a local resource that doesn't require a kludgy Automator-like process to get things to work. Maybe "set it and forget it" is a thing of the past.

 

[rehkram]

I have a different question: While TM backups from my M1 Mac to my Synology NAS work fine (speed is kinda okay-ish), I noticed that there's continuous IO happening on the NAS - even after the TM backup is finished. Looks like there's some kind of re-indexing of the existing backups happening? Anybody noticed this, too?

I had noticed it; given that a NAS is a computer dedicated to storage and non-trivial management thereof, I've become accustomed to it running related background tasks.

You can get visibility of what's running at that time on your Syno NAS via DSM using DSM > Main Menu > Resource Monitor > Task Manager, sort it by high- to low CPU percentage. You can drill down to see a task's subprocesses using the tiny right arrowhead in the left column. Then do a web search on task- or subprocess name (since the names are not always intuitive).

My 10 year old 1511+ is still going strong with around 75,000 hours of uptime on all five WD 2 TB drives. I'm using Synology Hybrid RAID, with 1 disk fault tolerance. 2 years ago one of the drives got flagged with sector errors and 'close to failure' by a SMART test (I run them manually every so often) so I replaced it with a similarly aged used drive of the same model. Much to my relief the system rebuilt itself successfully.

I've had no problem using TM on Big Sur with it. It's getting old now, time to get a new one probably.

 

[richsadams]

Even though they usually finish in 10 minutes or so, TM's wildly inaccurate calculated backup time estimates still worry me. I feel like they're a red flag for imminent failure.

So I'm thinking about switching to a Synology DS220j and a couple of Seagate IronWolf NAS HDDs. Should I expect to see the same issue?

It's always good to hear the personal experiences here, positive or negative.

TIA!

 

[ARMunix]

You can get visibility of what's running at that time on your Syno NAS via DSM using DSM > Main Menu > Resource Monitor > Task Manager, sort it by high- to low CPU percentage. You can drill down to see a task's subprocesses using the tiny right arrowhead in the left column. Then do a web search on task- or subprocess name (since the names are not always intuitive).

The IO on the NAS is definitely caused by the Mac, not some application running on the NAS: mds is going through all existing backups. Don’t know what it’s doing. It will eventually finish, but the more TM backups pile up, the longer this whole process is going to take.

 

[rehkram]

So the Mac is indexing it for Spotlight. You can tell macOS not to index that drive... or am I missing something?

mds and mds_stores constantly consuming cpu

mds and mds_stores are constant CPU consumers even when I think nothing should have changed on the hard drive. Activity Monitor: I have a circa 2009 Macbook Pro with lots of software installed ove...

apple.stackexchange.com

 

[ARMunix]

So the Mac is indexing it for Spotlight. You can tell macOS not to index that drive... or am I missing something?

But I want it to index the backup volume... that's not the point. The point is that it's doing it again after every new backup, revisiting all previous backups. I don't see any reason for that.

 

[rehkram]

I see. I have no requirement to index my backups so it's not a problem for me I guess.

 

[VintageMac]

Just read through some of this thread. Most of it is far over my head. Any understandable responses to my current situation? Genius Bar said my wife should do a complete backup of her MacBook Pro so they can they wipe it clean and then set it up so it is no longer married to my MBP with same Apple ID etc. Five hours ago I started the back up with Time Machine to a WD Passport via usb. At this point it has done 350mb out of 94gb. It seems to go in very small spurts, then not showing any change for a while. Please tell me this is not normal.

 

[HDFan]

So I'm thinking about switching to a Synology DS220j and a couple of Seagate IronWolf NAS HDDs. Should I expect to see the same issue?

Yes. Although it depends on the size of your backup. My 2 TB backup to a Synology running IronWolfs was unusable. Slow, then gets corrupted.

I'm using iDrive for cloud backups,

iCloud is not a backup. A 3-2-1 backup strategy is recommended.

 

[dim_]

So the Mac is indexing it for Spotlight. You can tell macOS not to index that drive... or am I missing something?

mds and mds_stores constantly consuming cpu

mds and mds_stores are constant CPU consumers even when I think nothing should have changed on the hard drive. Activity Monitor: I have a circa 2009 Macbook Pro with lots of software installed ove...

apple.stackexchange.com

No, that is no longer possible since at least a few years, see e.g. https://eclecticlight.co/2016/06/20...otlight-indexing-on-your-time-machine-backup/ . If you open Spotlight preferences, and drag your Time Machine backups folder onto the Privacy list, it literally says "“Backups of blah” is a Time Machine backup folder. You cannot add it to the privacy list" ! Apparently the only option is to turn off Spotlight globally. But then the question is whether Time Machine backups will be accessible in a normal fashion.

But I want it to index the backup volume... that's not the point. The point is that it's doing it again after every new backup, revisiting all previous backups. I don't see any reason for that.

Indeed, I also see exactly the same issue, since updating from 11.2 to 11.3. It persists in 11.4.

See also this thread: https://forums.macrumors.com/thread...k-bs-11-3-edit-still-present-in-11-4.2293652/

 

[rehkram]

Gotcha. Looks like I'm in for an interesting time in a couple of days when I update from Catalina to Big Sur 11.3.1 using OCLP. Just waiting on a 1 TB OWC replacement drive to install in my rMBP 2012 before installing Big Sur.

 

[strapper2021]

Also late to the party here but I believe I have reached the end of the line when it comes to TM. I have read every post and appreciate all of the suggestions, to get it to work, but no thanks. Whatever happened to "It just works"?

Prior to updating my iMac (Retina 5K, 27-inch, 2017) to Big Sur, there were no problems with TM backups to my 2TB Time Capsule (the latest, last model Apple made). After updating TM backups became unbearably slow and eventually would never finish. They still worked normally on my wife's iMac and my MacBook, both still on Catalina.

After following Apple's (and other's) recommendations I wiped the Time Capsule drive and reformatted it to APFS. The initial backups for all three Macs worked great and finished overnight. My hopes were high but immediately dashed since then as TM backups for all three are now unbearably slow. The first one since the full backup on my iMac says it will take 8 hours to back up a little over a gig. Since the initial update I've only opened mail and worked on a few websites. I know I have not generated more than 1 gig of new files. No idea why it's so large.

All of that said, I have no interest or the time to jump through all of the hoops suggested here to get an NAS or regularly connected drive to work with TM. It's clear in my mind that since dropping Time Capsules that Apple is no longer interested in backup support of any type. In this day and age of ransomware, malware and more that seems pretty incredible. Even to restore a failed HDD or SSD requires a good backup plan.

I've set up iDrive for all three of our Macs and iPhones for emergency purposes, but would still like a "hard copy" locally since recovering files stored in the cloud can take a very long time.

Has anyone moved to another option? I have Super Duper, but it doesn't play well with Big Sur, at least yet and I don't want to have to set up three different backup drives. I'd be willing to set up an NAS if it doesn't involve TM.

Thanks for listening, welcome any thoughts and TIA!

I've been having similar problems with my TM backups ever since updating to Big Sur. I have been backing up an iMac and 2 MacBook Pros for several years to my Synology NAS with zero problems and excellent performance. The iMac is connected directly to my Velop Router via ENET cable and the laptops connect via wifi.

Once I updated the iMac from Mojave to Big Sur, everything went to hell. I am having all of the same issues reported in this thread, including backups taking forever, "preparing backup" doesn't always finish and backup fails, etc.

I tried deleting the entire sparsebundle for the iMac on the NAS and created a new backup and then incremental backups. I am still having the same issues as before.

I even bought a WD MyPassport, connected it directly to the iMac via USB to help trouble shoot the problem. The drive formatted as APFS and TM backups work great - fast, reliable and no issues.

It pisses me off that I updated to Big Sur and now have all these issues with TM. With multiple computers, the NAS makes the most sense for me. It makes no sense to purchase stand alone HDD or Flash Drives to backup each machine, especially the laptops which move around the house.

I don't understand all of the details wrt SMB, AFPS, etc. that are mentioned by the smart people in this thread. So I would welcome any suggestions, including dropping TM and purchasing some other backup solution.

Thank you for any suggestions you can offer. Sleepless in Seattle until I get my backups working properly.

 

[petterihiisila]

It's been a while, I thought I'd post an update. Things have somewhat improved.

Time Capsule (sparsebundle) and a local disk backup (USB3 direct) both now work fast enough that I could keep the hourly backup going, no need to use TimeMachineEditor. Both are using vanilla Big Sur settings: encrypted APFS. The local disk usually gets the job done within 5-20 minutes, which is OK.

I've decided to not use the NAS target for now, even though the performance is acceptable. It still sometimes takes more than an hour to complete, even for just a few GBs. What's worse is that Big Sur often has difficulties ejecting the NAS disk without a reboot, which means that I can't freely undock my M1 Macbook Air when I want to move it. Yanking off a mounted NAS drive will make the Mac panic a few minutes later. Force-ejecting a Time Machine sparsebundle is a recipe for data corruption. Ejecting local disks is much more reliable, so that's what it's got to be then.

As a long-term solution I've been considering USB3/C enclosures with a 4 TB SATA SSD, or maybe an NVME drive, but they're still annoyingly pricey at 4 terabytes. They are very small, easy to attach below the desk. And of course completely silent.

On my other Mac, I've got fewer files and a 500 MB Samsung T5 SSD as a Time Machine target is big enough. It performs very well. It usually takes just 1-2 minutes to complete a backup cycle there. Would be nice to have the same Time Machine performance on the primary computer. But for now, the local USB3 HDD will have to suffice. At least it works well within the hourly cycle now.

 

[HDFan]

I've been considering USB3/C enclosures with a 4 TB SATA SSD, or maybe an NVME drive,

Why an SSD? Except in some special circumstances a hard disk is much more cost effective (you can get a much larger one which will keep more history for the same price), and hard disk reliability can actually be better than that of an SSD. You initial backup is slower, but who cares - it just runs in the background.

 

[petterihiisila]

Why an SSD? Except in some special circumstances a hard disk is much more cost effective (you can get a much larger one which will keep more history for the same price), and hard disk reliability can actually be better than that of an SSD. You initial backup is slower, but who cares - it just runs in the background.

So you prefer hard disks. Username checks out. ?

I don‘t think I’ll be able to convince you that I don’t prioritize months-long history, but that I do prioritize quick hourly updates, so that I can undock my M1 Air pretty much anytime I want to. Currently testing an SSD as a backup target, and the update usually last less than a minute. Pretty happy with it so far.

 

[HDFan]

Username checks out.

Never thought of that, actual intent at the time was for video quality. Not a great choice.

 

[lacek]

It is interesting to hear that I am not the only one with this problem. In my opinion this is something that Apple needs to fix. Switching back to "stone age" by performing manual backups on USB disks is pretty much the last thing I want do do, and since I am talking about not only my machine but also others in this network it would be a logistic nightmare.

On day 3 with my "optimised" setup (based on the observation that it is really the frequently changing small files bringing my backups practically to a halt I blacklisted all caches created by browsers and messengers) all backups have gone through so far though taking considerably longer than they should. It will be interesting to see how this "survives" real changes, like when apps are upgraded or the next MacOS update comes.

Hi,
These days Time Machine does not work reliably over the network. One option is to rsync your drive to a NAS (one can use cp -T on nas to create "incremental backup that holds entire tree structure of your disk" So under 2021-06-10-1224 you get your data from 10 June 2021 at 1221, but only the "differences" take the actual space.

A better solution would be to use a ZFS filesystem on NAS and its snapshots to provide versioning of files. On Mac you have APFS file system which also has snapshots feature. Just create a snapshot before backup, make a diff with previous snapshot, send files in the diff to the file server and you are done.