Time Machine backups painfully slow on NAS (smb)

[mbert]

After having upgraded to Big Sur I keep having time machine problems. I am using a Synology NAS to which I connect via SMB. This setup became necessary with Mojave as then I wasn't able to use AFP anymore, and it had been working OK ever since.

After the Big Sur upgrade I was unable to get this into my existing backup. So I renamed it and created a new one (about 12 hours for about 400 GB). That went along just fine. For the first few days things were OK. It spun out of control when the upgrade to 11.1 (and hence a large change that needed to be backed up) came: backups could not be completed during the day, and at night (despite of power nap) they'd eventually get interrupted, so that on the next day they needed to get continued. When now triggering a time machine backup it takes more than an hour before the actual backup starts (sizing etc.), and when in progress it is painfully slow.

It looks like what really steals the time are all the small files, stuff under /usr/local (brew) and preferences under ~/Library. When tracing the time machine logs I can see that it does around "~- MB/s, 1.49 items/s". As a consequence, the backup never catches up with the changes I produce when using my machine during the day (like Emails coming in, caches changed etc.).

I have noticed that during backup a directory '/Volumes/Backups of <my-machine-name>' is created, and browsing it, e.g. in terminal, is again extremely slow. The directory contains a growing number of directories named like '2020-12-14-203649.interrupted'. The one like '2020-12-23-081332.inprogress' never finishes and becomes '.interrupted' next morning.

Of course I could do another backup from the scratch, but I feel pretty certain that after that I will eventually end up with the same problem I have now.

Since the initial backup is reasonably fast I don't think that my SMB connection to the NAS is the problem here. But something is definitely wrong. Anyone got an idea?

P.S.: as recommended by Synology I have set "signing_required=no" in /etc/nsmb.conf long time ago and never had to change it. Also yesterday I tried to speed up SMB by setting "com.apple.desktopservices DSDontWriteNetworkStores" to "TRUE". Neither helps.

 

[mbert]

Another P.S.: when running 'smbutil statshares -a' on the command line, it hangs for minutes. In the end it fails: "stat_all_shares : stat_share() failed for <volume-name>", and at the same time (caused by this?) my backup failed.

 

[RiderX]

I use AFP and Time Machine works much better than on Catalina. AFP is “depreciated“ but just works. I also set up a completely fresh backup bundle with the new Apple file system.

There is one bug though: if a backup gets canceled, you should relaunch Finder. Finder sometimes blocks unmounting the image. Other than that, it is rather fast over a 1 GBit ethernet connection.

(Mac Mini M1)

 

[mbert]

I use AFP and Time Machine works much better than on Catalina. AFP is “depreciated“ but just works. I also set up a completely fresh backup bundle with the new Apple file system.

I used to use AFP, but it stopped working when I upgraded to Mojave (or was it Catalina? Can't remember). The only way I got it working was by switching to SMB.

 

[RiderX]

No, afp is still working on Big Sur. I use a crappy WD NAS which does support afp. I tried SMB with Catalina and had only problems.

 

[mbert]

No, afp is still working on Big Sur. I use a crappy WD NAS which does support afp. I tried SMB with Catalina and had only problems.

Yes, I understand that it still works - in principle. However with the NAS I use I no longer got it to work after the abovementioned OS upgrade. In my opinion this is a shame, in particular, since the SMB support leaves much to be desired.

 

[RiderX]

I don’t know Synology NAS, but it is Linux as well and should therefore support afp. You may play around with the settings. The backup bundle we use over ethernet, consist of many small files and should not make any problems. In theory SMB is better, but the old AFP “just works”.

 

[mbert]

I don’t know Synology NAS, but it is Linux as well and should therefore support afp. You may play around with the settings. The backup bundle we use over ethernet, consist of many small files and should not make any problems. In theory SMB is better, but the old AFP “just works”.

Yesterday I deleted my existing backup and started a new one. It took 14 hours, so nothing spectacular. Today, as new slices came in, the backup stalled again, progress of a few bytes per minute. I then changed my configuration to use AFP instead of SMB which seems to be possible again (after I had not been able to get it going before) and triggered a backup (on top of the existing backup) which was of about 900 MB size. It went just fine until just over 800 MB, now it's stalled again.

Looks like this is not a problem related to my SMB setup. By the way - while a backup is running, is it considered to be normal that the directory '/Volumes/Backups of <my-machine>' is browsable extremely slowly? I had only noticed it when investigating this problem, hence I cannot really compare.

 

[BrianBaughn]

Been following this and I'm wondering…when is the last time either of you successfully restored a macOS system from a Time Machine backup located on a NAS?

 

[petterihiisila]

Since the initial backup is reasonably fast I don't think that my SMB connection to the NAS is the problem here. But something is definitely wrong. Anyone got an idea?

Here's a "me too" to this issue. I had better luck with HFS+ NAS earlier this Fall — it's slow but tolerable. APFS & NAS is not usable with M1 / Big Sur 11.1, the backups don't finish fast enough to catch up with new updates.

But now this HFS+ drive is wiped and I haven't yet tried to recreate the sparsebundle as HFS+, it's unclear what the process is today to make it discoverable as a TM target. And frankly, I'm not sure if it even helps.

For the time being I've had to abandon two NAS targets and use a local direct-connect USB3 drive instead.

FWIW, maybe try to what happens if you use AFP to HFS+ sparsebundle? And if you do, please share the results.

 

[mbert]

Been following this and I'm wondering…when is the last time either of you successfully restored a macOS system from a Time Machine backup located on a NAS?

A full system? Never. I know what this means. To my defence I have to say that each of the two times I got a new machine I needed to transfer ASAP because the respective old one had become unusable - but still could be used as a storage source which was 2 instead of probably 10 hours if I had used the TM backup.

However I have restored accidentally deleted data from my TM backup several times without any flaws.

 

[petterihiisila]

Been following this and I'm wondering…when is the last time either of you successfully restored a macOS system from a Time Machine backup located on a NAS?

I've done that twice in 2020. Why do you ask? Has it been unreliable for you? Yes, it's slow and always has been, but it gets the job done, given time. With Ethernet it's fine. With Wifi ... well, better have a really small system to restore.

The initial backup and first restore are both as fast as expected. But with Big Sur, APFS and M1 Macs: one, two or all of those factors somehow make incremental backups very slow.

 

[BrianBaughn]

I've done that twice in 2020. Why do you ask? Has it been unreliable for you? Yes, it's slow and always has been, but it gets the job done, given time. With Ethernet it's fine. With Wifi ... well, better have a really small system to restore.

The initial backup and first restore are both as fast as expected. But with Big Sur, APFS and M1 Macs: one, two or all of those factors somehow make incremental backups very slow.

That's good. Just checking. I don't have anyone doing it that way but good to know it still works.

 

[mbert]

The initial backup and first restore are both as fast as expected. But with Big Sur, APFS and M1 Macs: one, two or all of those factors somehow make incremental backups very slow.

Since I don't have an M1 we may probably be able to take that from the list.

 

[petterihiisila]

Some progress. I created an encrypted HFS+ sparsebundle into a Time Capsule using Disk Utility. Then I mounted it via Finder and used "sudo tmutil setdestination" to tell Time Machine to start using it as a backup target.

So far the performance has been the same as with Catalina, which means "good enough to use it". A small incremental backup takes 5 mins over Ethernet and 11 mins over WiFi. That's about normal.

The only side effect is that currently Time Machine treats this target as a local folder, which means that I have to manually mount the sparsebundle to enable backups. I don't mind doing that, because it gives some more control over when backupd will be running. I don't want it to run over Wifi constantly; that'll very easily corrupt the sparsebundle if the computer sleeps or disconnects in the middle of a backup cycle. Time Machine will keep making hourly local snapshots anyway, so I'll offload those to the NAS when docked.

If you've got enough space in your Synology, try that and see if HFS+ makes a difference? Just set the old backup bundle aside for the time being.

I haven't had a big snapshots to backup yet. Let's see if the lucky strike continues when it's time to back up one of those.

Btw, I excluded /opt/homebrew from backups, since it's trivial to restore a homebrew installation if it gets toasted. That way there's fewer tiny files for Time Machine to check.

 

[mbert]

I haven't had a big snapshots to backup yet. Let's see if the lucky strike continues when it's time to back up one of those.

Btw, I excluded /opt/homebrew from backups, since it's trivial to restore a homebrew installation if it gets toasted. That way there's fewer tiny files for Time Machine to check.

I, too, have started excluding folders from the TM backup, like all the caches under ~/Library, /opt/local (ports) and /usr/local (brew). I don't really like this, but for the time being I hope to work around the problems. So far I have been able to finish all backups. Not ideal, and this should really be fixed by Apple.

 

[rehkram]

OP, do you still have AFP enabled on the NAS or did you disable it? If AFP is disabled, try re-enabling it. Leave SMB enabled.

Like you I switched to SMB a while ago. After a macOS update, I dunno which one, Time Machine backups stopped working. I got it working again (eventually) by having both SMB and AFP enabled on my Syno 1511+.

My SWAG on that was Time Machine was relying on AFP for signaling, and SMB for data transfer. I may be barking up the wrong tree but it's been solid again since re-enabling AFP on the NAS

 

[mbert]

I cannot switch AFP off on the NAS atm, because there are machines in my network still needing it. That setup had worked well - until Big Sur came around. And now, even when using AFP, I am still having problems, even though they seem less dramatic...

 

[petterihiisila]

I cannot switch AFP off on the NAS atm, because there are machines in my network still needing it. That setup had worked well - until Big Sur came around. And now, even when using AFP, I am still having problems, even though they seem less dramatic...

20 hours in between, the next backup to a NAS / HFS+ sparsebundle took 30 mins over Wifi, which is similar to what it's been for the past 10 years.

I'm becoming quite convinced that APFS is the bottleneck in this setup, not AFP/SMB. Unless it's the combination of both.

To be sure, how can I check, whether the Time Capsule connection is via SMB or AFP? I'm using whatever the default is. Firmware 7.9.1.

 

[DimaVR]

I have two nas devices and all were fine and she Big Sur beta 1 came out 1 drive refused to back up. Search my name and thread I posted like 1-2 pages with my issues. Called TM issues

 

[Gogol]

Since Big Sur update I switched back to external USB disks on my desktop and MacBook.
APFS and snapshots changed things with time machine backups over a network. Time Machine backups became slower and slower during the years. Especially cleaning up or clearing free space came almost to a halt.
So now I do local disk backups on windows and Mac for fast bare metal restore (Time Machine and Acronis True Image) and my files are synchronised one way to the Cloud (Google Drive in my case) as a last resort in case bare metal restore doesn't work.
I still have two Synology NASses, but I don't think I will buy new ones.

 

[mbert]

It is interesting to hear that I am not the only one with this problem. In my opinion this is something that Apple needs to fix. Switching back to "stone age" by performing manual backups on USB disks is pretty much the last thing I want do do, and since I am talking about not only my machine but also others in this network it would be a logistic nightmare.

On day 3 with my "optimised" setup (based on the observation that it is really the frequently changing small files bringing my backups practically to a halt I blacklisted all caches created by browsers and messengers) all backups have gone through so far though taking considerably longer than they should. It will be interesting to see how this "survives" real changes, like when apps are upgraded or the next MacOS update comes.

 

[HDFan]

Certainly would be interested in hearing from someone who has successfully run TM NAS backups. I could never get an incremental 4 TB drive backup to either a Synology (1GbE) or QNAP (10GbE) to work in a reasonable timeframe. Worse yet after a month or 2 they always got corrupted and I had to start over again. I was not using sparsebundles though.

 

[bknaepen]

I have observed the same behaviour. Since upgrading to big sur, time machine (smb + AFPS) has become really slow.

 

[SjoukeW]

I am a software developer and I had a 'lot' of small files in the Time Machine backup. It had real struggle to get to the end. I found on the mighty internet that the backup is throttled so it doesn't use so much cpu etc. while doing its work in the background. This did prevent it from completing in my case. It really struggles with a lot of small files, I had millions of files ......
There is a terminal command which can change the throttle behaviour and make it faster. It did really work for me to get the initial backup done.
Here is a link: https://all-about-apple.com/speed-time-machine-backups-terminal-trick/
I hope it helps.