Resolved Is it normal for my Time Machine drive to be filling up so fast?

[2024]

I have an M3 iMac with a 1TB internal SSD that is about half full. In early 2024 I started using a 2TB WD HDD for Time Machine (so about four times the data on the internal drive). In a year and a half the TM drive was so full that I couldn’t use it anymore. I bought a new 8TB WD HDD to replace it, thinking I'd get 5-6 years of TM backups out of it, but in only eight months it’s already over half full (5.2TB used).

Is this normal? I’m a light user; mostly Safari, Numbers, Pages, etc. Nothing more data-heavy than downloading the occasional song and taking a few photos and/or 1-2 minute videos now and then.

The TM drives were formatted by the iMac, so they’re APFS. I back up manually, twice a week, because I don’t feel the need to do it more often. I don’t understand how they can be filling up so fast. When I open the TM drive in Finder, it shows each semi-weekly backup as 500+ GB (in the Size column). However, that obviously can’t be each backup amount itself. I’ve excluded nothing from TM in Settings.

I’m very puzzled by all this. Does anyone have any thoughts?

When I add or move a photo, does the entire photos database get recreated in TM each time? My snapshots are removed after running TM, so I don't think they're somehow being backed up subsequently (if that's even possible).

I know about CCC and Super Duper, so they don’t need to plugged here. I want to know about my TM.

 

[FreakinEurekan]

Time Machine backs up what’s changed; if you add a photo it’s not going to re-backup the whole library but just that photo.

One data killer I’ve seen with Time Machine is “scratch” data, which can take a lot of forms but essentially it means data that’s only there temporarily. Sometimes it’s as simple as excluding your Downloads folder from the backup, though some apps (e.g. Adobe) will use a specific folder for scratch data and you should exclude that too.

Personally I recommend setting backup to Auto and either leave it on the Hourly setting, or Daily at the longest. You won’t really use much if any more disk space on TM and possibly less in the long run, because your backups will have better granularity and make it easier for TM to prune as necessary. And it’s an iMac, so there’s no reason to not just leave the TM drive connected full-time.

I know about CCC and Super Duper, so they don’t need to plugged here.

I hear ya… CCC and Super Duper are good apps but they are NOT BACKUPS. They are clones, which is a completely different (not invalid - but different) solution.

 

[Fishrrman]

"CCC and Super Duper are good apps but they are NOT BACKUPS. They are clones,"

This statement is ridiculous on its face.

From time to time, I've messed up a file, and needed to "get back to where I once belonged".

I reached for my CCC backup, found the file I needed, and things were working again.

I've also done this with entire accounts. It works. Running again, when (before) I wasn't.

A "clone" absolutely IS "a backup".
It's a backup of what your drive was like the moment you cloned it.
For me, that's about all I ever need.
And CCC can be set to retain changed versions of files, as well. It's callled "safety net".
Obviously you don't know what the app can do.

 

[BigBlur]

In a year and a half the TM drive was so full that I couldn’t use it anymore.

Did it give some sort of an error? It should start deleting the oldest backups.

Any chance you're using virtual machines? That is another thing that can take up a lot of space. If you've got a 100 GB virtual disk and make a small change inside the VM, it's going to backup the whole 100 GB virtual disk again unless you exclude it from the backup.

 

[!!!]

TM should automatically delete older backups to make room for new backups. You can delete individual backups as well.

I'd be more interested in finding out what's filling your drive so much. There used to be a free tool called TimeTracker, made by the same people as Pacifist. However it's no longer available and it only worked on HFS-formatted TM drives. BackupLoupe is a paid alternative, though I have not used it.

Some basic questions, though:
Do you use a virtual machine (VM)? They typically bloat TM backups due to the abstraction between the VM's FS and the underlying disk image.
Do you edit large movies?
Do you frequently move large files, or move or rename folders containing large files?
Do any of the websites you visit use local storage extensively?
Do you have any large apps that are frequently updated, particularly games?
Have you looked through your ~/Library folder? Anything stick out as particularly large? There may be a runaway app writing excessive amounts of data.

Other than that, there are some folders inside the hidden system /private/ directory that, for some reason, are backed up despite being caches or log files.

 

[svenmany]

My issue with TM was the spotlight indexes which got very, very large. I believe it is a bug and it forced me to stop using TM.

After my file changes were copied to the TM drive, Spotlight indexed everything, adding to whatever index was there before. All of that content was then put into an APFS snapshot on the TM drive. Those snapshots are what you see when you look at a TM drive in Finder. So each snapshot gets a spotlight index that is larger than the last. The bug I encountered related to just how large those indexes got. In fact, every time a TM drive was mounted, a reindexing of the oldest snapshot was done and the results added to the index in the root of the drive (which would make its way into the next snapshot). Since I reboot my computer regularly, this was a significant space hog and a constant wear on the drives.

To see if you are having this problem, you'll have to inspect the spotlight index using Terminal. Finder does not present the actual content of the drive's filesystem, only the snapshots there. You would also notice this problem if after a reboot, spotlight is very active for more than 10 minutes. For me, Spotlight would work for hours on the Time Machine volumes after every reboot.

 

[hobowankenobi]

TM should automatically delete older sets, so as a volume fills, it should not stop working, unless there is a problem or some file(s) are so large that even with the minimum versions kept (there is not enough free space to complete).

one of my gripes about TM has always been that there is not enough granular control. I would, for example, like to say that I want to keep 3 versions of some files, and only 1 of some, and perhaps 10 versions of others. All that is set automatically based on free space. More free space = more history.

One can consider excluding or limiting what is getting backed up too, to save space for more history of the most important/unique files.

Some will land on a more nuanced approach:

1. A snapshot or a full system clone on some predetermined schedule.
2. Continuous backup of important working files that change most often.

TM is pretty good for 2, but less useful for 1. Lots of other tools that can handle the system snapshot. With that in place (on another drive) TM could still do well with continuous backup of specific directories.

 

[2024]

Time Machine backs up what’s changed; if you add a photo it’s not going to re-backup the whole library but just that photo.

One data killer I’ve seen with Time Machine is “scratch” data, which can take a lot of forms but essentially it means data that’s only there temporarily. Sometimes it’s as simple as excluding your Downloads folder from the backup, though some apps (e.g. Adobe) will use a specific folder for scratch data and you should exclude that too.

Personally I recommend setting backup to Auto and either leave it on the Hourly setting, or Daily at the longest. You won’t really use much if any more disk space on TM and possibly less in the long run, because your backups will have better granularity and make it easier for TM to prune as necessary. And it’s an iMac, so there’s no reason to not just leave the TM drive connected full-time.


I hear ya… CCC and Super Duper are good apps but they are NOT BACKUPS. They are clones, which is a completely different (not invalid - but different) solution.

Thanks for your ideas. It's reassuring to hear that the entire photos database isn't copied every time I add a single picture.

I don't think scratch data are an issue. About all I ever download is an occasional pdf, which then goes into the appropriate folder; or a screenshot for temporary reference.

Re leaving TM on, I prefer to keep it air gapped because we get some electrical fluctuations here, and (mainly) because the TM drive is annoyingly loud and gets on my nerves.

Thanks again. I'll think more about what you've mentioned.

 

[2024]

Did it give some sort of an error? It should start deleting the oldest backups.

Any chance you're using virtual machines? That is another thing that can take up a lot of space. If you've got a 100 GB virtual disk and make a small change inside the VM, it's going to backup the whole 100 GB virtual disk again unless you exclude it from the backup.

The first TM drive deleted a few months' worth of files, but then said the drive was too full and could no longer be used with TM (something to that effect). The second drive is working fine but seems to be filling up very fast; hence my post.

I've never used a VM on the iMac. Thanks for the warning about it.

 

[2024]

Do you edit large movies?

No

Do you use a virtual machine (VM)? They typically bloat TM backups due to the abstraction between the VM's FS and the underlying disk image.

No

Do you frequently move large files, or move or rename folders containing large files?

Only of MB, not GB, size. I do have 100+ notes in the Notes.app, some of which I edit pretty frequently, but they're mostly just text.

Do any of the websites you visit use local storage extensively?

No

Do you have any large apps that are frequently updated, particularly games?

No

Have you looked through your ~/Library folder? Anything stick out as particularly large? There may be a runaway app writing excessive amounts of data.

That will take a while to peruse. I'll try to get to it tomorrow.

Other than that, there are some folders inside the hidden system /private/ directory that, for some reason, are backed up despite being caches or log files.

I don't know how to access that.

Thank you for all the suggestions.

 

[2024]

My issue with TM was the spotlight indexes which got very, very large. I believe it is a bug and it forced me to stop using TM.

After my file changes were copied to the TM drive, Spotlight indexed everything, adding to whatever index was there before. All of that content was then put into an APFS snapshot on the TM drive. Those snapshots are what you see when you look at a TM drive in Finder. So each snapshot gets a spotlight index that is larger than the last. The bug I encountered related to just how large those indexes got. In fact, every time a TM drive was mounted, a reindexing of the oldest snapshot was done and the results added to the index in the root of the drive (which would make its way into the next snapshot). Since I reboot my computer regularly, this was a significant space hog and a constant wear on the drives.

To see if you are having this problem, you'll have to inspect the spotlight index using Terminal. Finder does not present the actual content of the drive's filesystem, only the snapshots there. You would also notice this problem if after a reboot, spotlight is very active for more than 10 minutes. For me, Spotlight would work for hours on the Time Machine volumes after every reboot.

I've never used terminal, so someone would basically have to lead me by the hand to do what you suggest. I'm willing to give it a try though.

How would I tell if spotlight is working on the TM drive? Somewhere in Disk Utility?

I'm still a novice with macOS, so you'll have to bear with me when things start getting technical. 😉

 

[2024]

One can consider excluding or limiting what is getting backed up too, to save space for more history of the most important/unique files.

I exclude nothing at this time. Probably 95% of my data has not changed since I copied it onto the iMac originally. That's partly why I'm so puzzled by what I'm seeing.

 

[gilby101]

I have an M3 iMac with a 1TB internal SSD that is about half full. In early 2024 I started using a 2TB WD HDD for Time Machine (so about four times the data on the internal drive). In a year and a half the TM drive was so full that I couldn’t use it anymore. I bought a new 8TB WD HDD to replace it, thinking I'd get 5-6 years of TM backups out of it, but in only eight months it’s already over half full (5.2TB used).

How big are the snapshots on your TM disk? To find out use Disk Utility; in the menubar enable View > Show APFS Snapshots; select the TM volume. Next bit is very slow - particularly with hard drive. DU will show each snapshot and its "Private Size" - that is the disk space used by files unique to that snapshot. Are any of these large?

You change the columns to also include "Cumulative Size" - after another very ongoing wait this will show the size you would recover by deleting a snapshot and all the older ones.

You can also delete snapshots manually. Though you can do that in Disk Utility, it is easier in Finder. You could free up some space by deleing old snapshots - start with the oldest and work forward.

You also get BackupLoupe to get more detail - it indexes all files on each snapshot. IT will take a long time to scan your drive - leave it running overnight. It is great for identifying the new large files/folders in each backup. I think it is free for a trial period.

Add me to the list of people who say TM works best if the drive is kept connected at all times. There is background work which is only done after the backup is completed. If you really want to use a manual process, I suggest you connect the drive and start the backup in the evening and leave it connected over night - and don't let your Mac go to sleep.

 

[gilby101]

I hear ya… CCC and Super Duper are good apps but they are NOT BACKUPS. They are clones, which is a completely different (not invalid - but different) solution.

Sorry, but that is not true. Carbon Copy Cloner, by default, makes backups in a very similar way to TM. It uses snapshots, just like TM. Though name suggests it makes "clones", that is no longer true.

 

[2024]

How big are the snapshots on your TM disk? To find out use Disk Utility; in the menubar enable View > Show APFS Snapshots; select the TM volume. Next bit is very slow - particularly with hard drive. DU will show each snapshot and its "Private Size" - that is the disk space used by files unique to that snapshot. Are any of these large?

You change the columns to also include "Cumulative Size" - after another very ongoing wait this will show the size you would recover by deleting a snapshot and all the older ones.

I have no snapshots appearing at the moment. I'll let you know when a few appear. Thanks for the explanation.

You can also delete snapshots manually. Though you can do that in Disk Utility, it is easier in Finder. You could free up some space by deleing old snapshots - start with the oldest and work forward.

How would I find them in Finder? I thought they were deleted after being added to TM anyway.

You also get BackupLoupe to get more detail - it indexes all files on each snapshot. IT will take a long time to scan your drive - leave it running overnight. It is great for identifying the new large files/folders in each backup. I think it is free for a trial period.

Thanks, I'll take a look at that app.

Add me to the list of people who say TM works best if the drive is kept connected at all times. There is background work which is only done after the backup is completed. If you really want to use a manual process, I suggest you connect the drive and start the backup in the evening and leave it connected over night - and don't let your Mac go to sleep.

When TM finishes running, I always disconnect it right away. Sounds like that is the wrong thing to do.

I'll look into all the suggestions and will likely report back on Sunday when I have some time. Thanks again everyone.

 

[Eric5h5]

Before doing a backup, launch Terminal and type

tmutil compare -s

That will tell you everything that's changed since the last backup (will probably take a while to run); might give you some ideas.

As far as ejecting the drive.... I use Time Machine much like you, except every day or two, and the only issues I had were because of a bug during the 10.11 days when it would randomly decide to delete all backups and start over from scratch every few months or so. That was fixed in 10.12 and has never happened since. My current backup goes back 2.5 years, and before then I was using a smaller drive for a lot longer than that. Of course it filled up eventually, but it would just delete the oldest backups when needed.

Given that Time Machine by default operates hourly automatically, and given that you can sleep or shut down at any time regardless, it obviously has to be robust and can't be fiddly about being disconnected, or it would be useless. I don't necessarily disconnect immediately after a backup is finished, but the worst that can happen is that the mds process doesn't finish updating the index. The indexing for the first backup does have to finish 100%, but after that it's interruptible. It wouldn't hurt to use Activity Monitor and make sure the mds process hits 0.0% CPU after a backup at least once in a while, before disconnecting.

 

[hobowankenobi]

OP: As a less-experienced Mac user...you might consider checking out some of the details (and quirks) about TM. A good starter here. Check the links at the bottom for other write-ups. He knows his stuff, and explains well.

Also...consider excluding things you would live without should you need to restore a backup...to make sure you do have the things that are critical backed up.

 

[2024]

Before doing a backup, launch Terminal and type

tmutil compare -s

That will tell you everything that's changed since the last backup (will probably take a while to run); might give you some ideas.

As far as ejecting the drive.... I use Time Machine much like you, except every day or two, and the only issues I had were because of a bug during the 10.11 days when it would randomly decide to delete all backups and start over from scratch every few months or so. That was fixed in 10.12 and has never happened since. My current backup goes back 2.5 years, and before then I was using a smaller drive for a lot longer than that. Of course it filled up eventually, but it would just delete the oldest backups when needed.

Given that Time Machine by default operates hourly automatically, and given that you can sleep or shut down at any time regardless, it obviously has to be robust and can't be fiddly about being disconnected, or it would be useless. I don't necessarily disconnect immediately after a backup is finished, but the worst that can happen is that the mds process doesn't finish updating the index. The indexing for the first backup does have to finish 100%, but after that it's interruptible. It wouldn't hurt to use Activity Monitor and make sure the mds process hits 0.0% CPU after a backup at least once in a while, before disconnecting.

Thanks for the suggestions and tips. I'll be reading everything again to prepare for my next scheduled backup on Sunday (have some other things going on tomorrow and Saturday). I did get a chance to turn off sleep for several hours today with TM mounted, but didn't see anything noteworthy; mds was at 0% and nothing appeared to be happening on the TM drive the whole while. I wasn't running a backup, so I wasn't surprised by that.

 

[2024]

OP: As a less-experienced Mac user...you might consider checking out some of the details (and quirks) about TM. A good starter here. Check the links at the bottom for other write-ups. He knows his stuff, and explains well.

Also...consider excluding things you would live without should you need to restore a backup...to make sure you do have the things that are critical backed up.

Thanks you. I'll definitely read that link over the weekend. I've done a lot of thinking about what to remove from my internal drive, so a fair amount of old data from various Windows machines that are still backed up on other drives are already gone from the Mac. Everything I want to keep is currently on the Mac now and is going onto TM.

 

[svenmany]

Time Machine, by itself, seems to work flawlessly for me. When Spotlight gets into the mix, then all hell breaks loose. If you run your machine in safe boot mode, then Spotlight will not index anything and Time Machine will continue to function flawlessly. So Spotlight is not an integral part of Time Machine.

I had deleted all my TM volumes. In response to this thread I made a new TM volume and played around some. The behavior I'm getting now is different than before, but still a complete mess with respect to Spotlight's involvement.

People who never reboot and never unmount their TM volumes are at far less risk from Spotlight bugs with respect to Time Machine. Most of the indexing bugs that I've seen, happen when the volume is mounted where an existing backup already exists.

Just to be sure the space consumption is not due to runaway indexing, do the following in Terminal:

cd /Volumes/<name of your TM volume>/.Spotlight-V100
sudo du -sh

My new TM volume, which is not experiencing any issues, shows 11G, which is a completely reasonable size for the index. Even 20G would be reasonable. My computer's drive has an index of 11G (found at /System/Volumes/Data/.Spotlight-V100), but I might expect that the TM volume's would be larger since it might have to represented deleted content. It's pretty random if and when Spotlight indexes any particular snapshot. Some of my snapshots return results for searches and some don't (when using the TM browsing functionality).

 

[2024]

Just to be sure the space consumption is not due to runaway indexing, do the following in Terminal:

cd /Volumes/<name of your TM volume>/.Spotlight-V100
sudo du -sh

I'm leery of terminal, having never used it. I also don't now if I would understand how to interpret the results. I think I'll do some reading, and come back on Sunday evening to tell you what I see before I do my backup, and hopefully you can give me specifics on what to do to see if it worked right.

 

[2024]

I'm sorry I haven't been back. A medical issue in the family has kept me preoccupied; it may be a few more days before I can resume this. Things are okay, but I'm busy.

 

[2024]

Hi again. It's taken me more than a few days, but I finally have a little time on my hands. Last night I ran the trial version of BackupLoupe (thank you !!! for mentioning it!), which pointed me to the hidden Library folder. After getting more into the weeds in Finder, I found that "com.apple.KeyValueService-Production.sqlite" is at almost 88 GB, and in TM it's being updated every few days.

Does that kind of file size sound right? Is it possible it's being added anew to every TM backup? It certainly would explain why nearly every TM backup is 90-100 GB, though I'm not really sure if an instance of a "file" in TM (using Finder) is actually a complete file or not. I could be totally offbase, but it was a eureka moment when I found it.

The complete filepath is:
'/Users/<myusername>/Library/Daemon Containers/4521EE68-CCC2-4A62-B0D4-1F14A87FDAA1/Data/com.apple.kvs/com.apple.KeyValueService-Production.sqlite'

Any thoughts on this? If it is the issue, what would be the remedy?

 

[2024]

Oh, I ran "tmutil compare -s" and found nothing that jumped out at me, but I could well have missed something or misunderstood what I was looking at.. (What does "!" at the start of a line signify? I sure have a lot of them.)

Also, I tried "cd /Volumes/Time Machine WD Black 8TB/.Spotlight-V100", but got back "cd: too many arguments". So, thinking that might mean spaces are a no-no, I tried again using "cd /Volumes/Time_Machine_WD_Black_8TB/.Spotlight-V100" and got back "zsh: no such user or named directory: cd".

At that point I figured I'd stop while I was still ahead lol

 

[2024]

How big are the snapshots on your TM disk? To find out use Disk Utility; in the menubar enable View > Show APFS Snapshots; select the TM volume. Next bit is very slow - particularly with hard drive. DU will show each snapshot and its "Private Size" - that is the disk space used by files unique to that snapshot. Are any of these large?

I tried this, but couldn't get the TM drive to stay awake long enough. It always turns off after 5 minutes or so unless I'm actually doing a backup, even with "Put hard disks to sleep whenever possible" turned off in settings.