Resolved Time Machine backups to Linux server (netatalk+avahi-daemon) suddenly stopped working

[Mikael H]

Just putting this out there, primarily wondering whether this seems to be a common problem, and secondarily if anyone of you guys here has any additional tips for troubleshooting beyond what I've already done.

The problem
I've been running Time Machine backups to my Linux home server (currently running Ubuntu 16.04) for as long as I've had Macs. The solution is based on netatalk and avahi-daemon and has mostly been robust as long as the actual network connections have been good. Yesterday I rebooted my MacBook Pro (late 2013 retina), probably for the first time since installing Sierra, and my backups stopped working.

I've had a few error messages along the way, but the short of it is this:
"The network backup disk could not be accessed because there was a problem with the network username or password. You may need to re-select the backup disk and enter the correct username and password."

Troubleshooting steps
- When I connect to the share I use for TimeMachine using Finder, or, after mounting it, using Terminal, I can create directories and files on it.
- I created a second AFP share which I too can access for both reading and writing, but Time Machine keeps repeating that the credentials are wrong.
- I deleted all credentials related to anything AFP:// in my Keychain and recreated a connection, and that did not solve the problem.
- Based on (ancient) tips from Apple's support forums, I renamed the target of the AFP:// credentials, renaming "[servername].afpovertcp._tcp.local." to both "[servername].local" and to "[servername].local." This didn't solve the problem either.
- Based on other tips I tried using tmutil from the terminal to manually point to my backup target. It accepted the command but the subsequent backup failed.
- Attempting to switch between the two available AFP shares on the server results in a "Keychain error -25299 while creating a System Keychain entry for the username "[myuser]" and URL "afp://[username]@[servername]._afpovertcp._tcp.local./[shareName]".
- Again, removing the apparently faulty credentials from the keychain and retrying to establish a connection from within Time Machine fails, but after typing them in for Time Machine to use, I still have read/write access to the actual share using the same credentials.

---

Any tips for what I may be missing would be greatly appreciated.

 

[theluggage]

This has me worried. I use Ubuntu for Time Machine purposes but haven't switched my real Mac to Sierra yet - currently using the Ubuntu box that holds my TM backups dual-boot Sierra for testing so trying out Time Machine is... problematic. I'll find a way of giving it a try...

Found this - its about Synology NASs rather that Ubuntu but it might offer some clues/workarounds:
https://forum.synology.com/enu/viewtopic.php?t=120107

...problem seems to be preferring SMB over AFP (netatalk).

You mention avahi-deamon: when I installed Ubuntu 16.04, I just had to install netatalk and enable time machine for the volume in AppleVolumes.default and it Just Worked - I assuming its using avahi-daemon but I didn't have to do anything manually. Haven't tried this with sierra yet, but if you configured avahi manually, maybe that's the problem?

 

[Mikael H]

@theluggage: Thanks, great link! I don't run SMB at all on the server in question, but I don't think I've tried dismounting the AFP folder before attempting to assigning the share to TM as per the last couple of posts in the thread. I'll see what happens when I get home, and report back here.

---
UPDATE:
Well, that didn't work.
I got a bit hopeful when I saw that the current netatalk versions 3.1.10, while the one bundled with Ubuntu 16.04 was in the 2.x range, so I uninstalled Ubuntus native one and compiled the current one from sources (there's an excellent step-by-step instruction which only needs to be modified to account for the fact that 16.04 runs systemd where 14.04 run sysVinit), but apart from the handshake ("looking for backup disk") taking a lot longer, the end result is the same.

I guess I'll have to backup to USB drives for a while, while digging further into the issue.

 

[Mikael H]

Update 2:
I followed https://forums.macrumors.com/threads/selecting-location-of-timemachine-backups.1544990/ and created a sparsebundle on an SMB share instead. Backups are working, but I'll probably need to script the mounting of the network share for it to keep working. Not really a solution, then, but an acceptable workaround to avoid having to buy an additional drive.

 

[theluggage]

OK - my update, simplified sequence:

Virgin install of Ubuntu 16.04.1 44 bit desktop, as it comes, in a Parallels VM on my "real" Mac. Bridged ethernet, so it looks like its plugged directly into my home network. On this:

• Linux user is luggage password mypassword
• Create /path/to/timemachine and give luggage full r/w/x access to it
• sudo apt install netatalk
• append to /etc/netatalk/AppleVolumes.default:
  /path/to/timemachine "Virt Machine" options:tm
• sudo service netatalk force-reload

Then go to my Sierra test machine (10.12 release, clean install), fire up the Time Machine control panel:

• "Select Disk"
• "Virt Machine on 'virtubuntu'" shows up - select it
• Provide the user luggage and password mypassword

...and everything worked. Tried rebooting and it still works.

So, that's backing up from a clean-installed Sierra machine to a virgin Ubuntu install, which doesn't quite match your situation, but at least Apple haven't fundamentally broken Time Machine backup to netatalk.

If you did something more complex to install & configure netatalk (or went anywhere near avahi-daemon) then that might be the issue. Otherwise, could be the perils of upgrading your Mac OS machine rather than doing a clean install.

If you have Parallels (might be able to do it with the time-limited demo), perhaps you could install a virtual Sierra machine on it and see if that works with your server?

 

[Mikael H]

in a Parallels VM on my "real" Mac
...and everything worked. Tried rebooting and it still works.

Thanks, didn't even occur to me to try from a VM.

Now that I finally had some time to spare, I restored my original Netatalk version and my original settings for it on my server, and connected to a share from a relatively new VM in Fusion, and right enough, it worked flawlessly.
In other words I clearly have something wrong with my personal laptop's TM or Keychain settings, rather than this being a general problem between macOS Sierra and Linux based operating systems + netatalk.

 

[theluggage]

Thanks, didn't even occur to me to try from a VM.

VMs - how did we ever live without them?

In other words I clearly have something wrong with my personal laptop's TM or Keychain settings

...sometimes, there comes a time to do a clean install, archive your time machine backup and rebuild your disk for an OS change, rather than upgrading... its a pain, though.

 

[Mikael H]

Just an update:
I didn't want to reinstall my laptop as if it had been running some flavor of Windows. Unix-like systems are supposed to be fixable, right?

The issue lies in Time Machine not properly updating the system credentials for the target share. Following a hunch based on a forum post I encountered, I changed the host name on the Linux server, and after pointing at the exact same share but via the new server name, Time Machine started working immediately.

Technically this means that the problem isn't truly solved, but it was a satisfactory workaround since I only use the host name of the server within Time Machine, so it didn't break anything else.

The funny thing is that deleting all credentials related to the original host name from my system keychain and deleting my Time Machine configuration file did not solve the problem, so some information is clearly stored somewhere else too. Does the system user in whose context the backup service runs have its own keychain; possibly inaccessible to regular users?