Using tags to organise files long-term

[thiskiwicanfly]

Hi everyone,

I've recently decided to start using tags to better organise the files on my Mac - as I'm convinced it's a much better method to categorise files, rather than using text stored in the file's comment field (which I've been doing up until now).

However, before I start using my new custom tags in favour of using text in the comments field, I'm curious to know...

Where is the tag information kept? More to the point, if I ever copy a custom-tagged file to a new Mac - or if I ever need to recover my Mac from a fresh install and copy the tagged files back - will my new custom tags be lost?

I intend to tag a large number of files, so I don't want to find myself back at square one in a worst-case scenario.

 

[Brian33]

It appears that the tag information is kept as "extended attributes" of the file. I was curious so I used to some Terminal commands (see below) to display the extended attributes of a file to which I had assigned two tags via Finder. I assigned one of the default color tags (Purple) and one that I made up ("a-new-tag"). My test file was called "foo".

The -@ option of the 'ls' command will display the names of all the extended attributes of a file:

brian@macbook:~(0)$ ls -l@ foo
-rw-r--r--@ 1 brian staff 31 Oct 9 12:56 foo
com.apple.FinderInfo 32
com.apple.metadata:_kMDItemUserTags 65

From that it appears the tag info is stored in the "kMDItemUserTags" attribute. (You can ignore the FinderInfo attribute.)

The 'xattr' command can display the attribute names and values:

brian@macbook:~(0)$ xattr -l foo
com.apple.FinderInfo:
00000000 00 00 00 00 00 00 00 00 00 06 00 00 00 00 00 00 |................|
00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000020
com.apple.metadata:_kMDItemUserTags:
00000000 62 70 6C 69 73 74 30 30 A2 01 02 58 50 75 72 70 |bplist00...XPurp|
00000010 6C 65 0A 33 59 61 2D 6E 65 77 2D 74 61 67 08 0B |le.3Ya-new-tag..|
00000020 14 00 00 00 00 00 00 01 01 00 00 00 00 00 00 00 |................|
00000030 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000040 1E |.|
00000041

Notice that the UserTags attribute contains the strings "Purple" and "a-new-tag".

I expect that the tag info will be preserved for a file, even if it is moved or copied, as long as the method used is aware of extended attributes. Finder should certainly be fine. A quick test showed that the 'cp' (copy) command also preserved the tags. Any MacOS-aware program SHOULD preserve extended attributes, but there may be some that don't. Maybe someone with more experience will chime in here...

 

[thiskiwicanfly]

It appears that the tag information is kept as "extended attributes" of the file.

I expect that the tag info will be preserved for a file, even if it is moved or copied, as long as the method used is aware of extended attributes. Finder should certainly be fine. A quick test showed that the 'cp' (copy) command also preserved the tags. Any MacOS-aware program SHOULD preserve extended attributes, but there may be some that don't. Maybe someone with more experience will chime in here...

Thanks Brian33! Yes, I can see it certainly keeps the tag information intact as long as the file is retained in the macOS file system.

I also did a quick test by creating a text file on an external drive formatted in HFS+, then created a new custom tag on that file. I unmounted the external drive, and then removed the new tag from the tag list in Finder (Preferences -> Tags). When I reconnected the drive, the new custom tag was automatically re-created. So it works!

Interestingly enough, when I unmounted the external drive again, the new tag was automatically removed from the tag list. So it seems macOS knows that if I choose to remove a custom tag after the source drive has been removed, then it turns that new tag into "a dynamic tag" (it only exists while the source drive is connected), otherwise all new tag stays present on macOS all the time, even if the external drive has been disconnected. Amazing!

 

[JanErik75]

This is indeed great news.

However, if you use a cloud backup service, you need to make sure that custom tags are not stripped when files and folders are uploaded to their servers. You don't want to find out that your meticulously crafted tags are missing when restoring from a cloud backup. Backblaze is a major culprit in this regard. There might be others too.

So when choosing a cloud backup provider, make sure you choose one that keeps all your metadata intact.

 

[thiskiwicanfly]

So when choosing a cloud backup provider, make sure you choose one that keeps all your metadata intact.

That's a great point! I didn't think of that. Fortunately, in this case, most of my custom tags are for local files that I never intend on backing up to a cloud service, but gets backed up on a secondary USB (HFS+ formatted) drive instead.

But I'm really surprised and curious why cloud backup providers such as Backblaze would strip tags from uploaded files. Considering metadata file size storage is rather minimal, I couldn't imagine it's done to manage capacity on their servers. But I could be wrong.

Which makes me wonder what would happen if I ever decide to host the tagged files on a NAS (Synology) instead? I believe the majority of NAS devices use the BTRFS file system - so I would assume it would store file metadata?

 

[thiskiwicanfly]

This is indeed great news.

However, if you use a cloud backup service, you need to make sure that custom tags are not stripped when files and folders are uploaded to their servers. You don't want to find out that your meticulously crafted tags are missing when restoring from a cloud backup. Backblaze is a major culprit in this regard. There might be others too.

So when choosing a cloud backup provider, make sure you choose one that keeps all your metadata intact.

I've done a bit more research into "extended metadata" on the macOS and I've read that EXIF metadata that is associated with photo/image files is saved to the file itself, so is easily transferrable across OS platforms. Extended metadata for video files (also know as Descriptive Metadata) that stores information such as "cast", "producer", "director", "genre" etc, is also saved to the file itself, so should also be transferrable across services and other systems - as long as the service doesn't intentionally strip the extended metadata out in the copy process, such as some cloud storage backup services.

 

[JanErik75]

I apologise about the necro bump, but I realised I hadn't responded with all the info I've discovered.

I've done a bit more research into "extended metadata" on the macOS and I've read that EXIF metadata that is associated with photo/image files is saved to the file itself, so is easily transferrable across OS platforms. Extended metadata for video files (also know as Descriptive Metadata) that stores information such as "cast", "producer", "director", "genre" etc, is also saved to the file itself, so should also be transferrable across services and other systems

I am a hobby photographer and I take a lot of photos on a yearly basis. I am very happy that this is the case. If I send a digital image over to someone who is on Windows or Linux, I can rest assured that the EXIF metadata containing the camera settings are intact. I believe this is because EXIF is an open international standard, whereas OS extended attributes are not.

... as long as the service doesn't intentionally strip the extended metadata out in the copy process, such as some cloud storage backup services.

I have learned that Arq Cloud Backup preserves macOS Extended Attribute metadata such as Finder tags. They look very reputable, and use Wasabi as their storage provider. I haven't tried them yet, but I will test out their 30-day free trial and see how it goes.

You can also subscribe to just Arq as the backup client, and pair it with a cloud storage provider of your choice, but the complete package of Arq Cloud Backup looks very enticing to me.

Let's just hope they stay around for the long haul. They've been around since 2009 so that's a good sign.

 

[jeyf]

i dont know how much SMB supports extended file attributes. the apple version of smb is rumored not that functional.

if a file uses extended attributes and is put into a bundle it might be iffy one can pull it out with the attributes intact.

on my NAS box i often have issues where various apple files refuse to delete. Some situations are worse than others.

best of luck but its difficult to be positive here