Way to keep Time Machine from filling SSD?

[theorist9]

I'm planning to use an a 4 TB external SSD for Time Machine backups (that way accessing TM won't be excessively slow, like it is with my HDD). To maintain performance, you don't want an SSD to fill up, but that's eventually what TM will do—keep adding snapshots until there's space for no more:

There are various recommendations for the max usage on SSD's; Anand Shimpli gave 75% filled, so let's use that. Is there a way to get TM to start pruning backups when it gets to that point?

The alternative would be to prune the backups manually, but it would be tedious to do that regularly once the drive gets to 75%.

 

[BrianBaughn]

I don't know of any other options than the ones you see if you type "man tmutil" in Terminal.

I'm wondering if the general use capacity recommendations out there even apply to a TM SSD. Time Machine is a unique use.

 

[chabig]

I wouldn't worry about letting Time Machine manage itself. Engineers at Apple know more about SSDs than most of us.

 

[theorist9]

I wouldn't worry about letting Time Machine manage itself. Engineers at Apple know more about SSDs than most of us.

True, but Apple engineers don't always deliver configurations that provide optimum functionality, and then improve things subsequently. Maybe they simply haven't gotten around to optimizing TM for SSD's.

 

[BrianBaughn]

Maybe they simply haven't gotten around to optimizing TM for SSD's.

Maybe they haven't…maybe they have. Maybe you don't have to worry about it.

It seems logical that TM would be slower (but who knows how much) once the limit is hit and it has to perform deletions as well as the backups. Doing what you're asking for here would only make that happen earlier.

 

[bmansvk]

Create multiple paritions on your external drive. First one, used for TM, create with 75% capacity of entire drive. You can fill empty space with second partition (not used by TM).

It seems, that there is no way to limit APFS volume size (there is quota option, but TM will remove quota in selected TM volume). But partition formatted to 75% will work for you.

 

[theorist9]

f

Maybe they haven't…maybe they have. Maybe you don't have to worry about it.

It seems logical that TM would be slower (but who knows how much) once the limit is hit and it has to perform deletions as well as the backups. Doing what you're asking for here would only make that happen earlier.

As I tried to explain in my starting post, my specific issue with TM's slow speed (and my main reason for getting an SSD for TM) occurs when accessing TM (i.e. entering TM and scrolling through the snapshots). Background deletions, which happen only intermittently, would not affect that, unless you coincidentally happened to be accessing TM while it was performing a deletion.

So the fact that pruning would start happening earlier in the disk's life isn't a concern for me. Plus the only difference is that you get to the same steady state somewhat earlier.

 

[BrianBaughn]

I think the time I have spent reading and replying to this post is greater than the combined time I have spent in the last three years retrieving a file from either my Time Machine backup or my CCC backup. A couple of months ago I goofed up a Logic Pro project and needed the file from the previous day.

 

[theorist9]

Create multiple paritions on your external drive. First one, used for TM, create with 75% capacity of entire drive. You can fill empty space with second partition (not used by TM).

It seems, that there is no way to limit APFS volume size (there is quota option, but TM will remove quota in selected TM volume). But partition formatted to 75% will work for you.

I'd thought of doing that myself, but wasn't sure if that would work, because the two partitions can't see each other. E.g., I'm not sure if a 98% full 3 TB partition will be any faster than a 98% full 3 TB drive simply because, in the former case, there is a separate unfilled 1 TB partition next to it.

The alternative would be to create separate volumes instead of separate partitions, since volumes do share more than partitions. Of course, this is purely speculative on my part. I suppose I could email one of the SSD manufacturers and ask them.

 

[Thessman]

There is a utility called time machine editor, with which you can adjust the interval between backups and more.

 

[theorist9]

There is a utility called time machine editor, with which you can adjust the interval between backups and more.

There's a couple of utilities like that on the App Store. They only seem to adjust backup scheduling, not the %fill (or space available) at which pruning starts.

 

[NoBoMac]

Was curious about this for you-know-what and giggles sake, and took a look at the diskutil command. Might be able to format the drive as APFS and add a volume to the container that is limited in size.

Have not tested this myself, so some experimenting might be needed here.

If you do a "diskutil apfs addVolume" from command line, will get the short help for that. In the examples you will see: "diskutil apfs addVolume disk5 APFS Foo3 -quota 10g -reserve 5g"

So going off that, "diskutil listFilesystems" output, and the man page for diskutil (for roles list), something like the following might get the job done:

diskutil apfs addvolume diskN APFSX TimeMachine -quota NNGB -role T

NNGB is how many gigs you want to limit the volume/partition to. Quote: "GB is power of ten gigabytes where the multiplier is 1000000000 (1 x 10ˆ9)"

APFSX makes it case-sensitive.

Role of T is for Time Machine and found under "addVolume" in the man page.

diskN is the disk number for the APFS container returned from "diskutil list" command. In the following example, "disk3" is the APFS disk/container I currently have mounted (which happens to have a Time Machine volume).

/dev/disk3 (synthesized):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      APFS Container Scheme -                      +4.0 TB     disk3
                                 Physical Store disk2s2
   1:                APFS Volume ⁨TM01Misc⁩                460.0 GB   disk3s1
   2:                APFS Volume ⁨TM01Secure⁩              105.3 GB   disk3s2
   3:                APFS Volume ⁨TimeMachine01⁩           457.7 GB   disk3s3

 

[theorist9]

Was curious about this for you-know-what and giggles sake, and took a look at the diskutil command. Might be able to format the drive as APFS and add a volume to the container that is limited in size.

Have not tested this myself, so some experimenting might be needed here.

If you do a "diskutil apfs addVolume" from command line, will get the short help for that. In the examples you will see: "diskutil apfs addVolume disk5 APFS Foo3 -quota 10g -reserve 5g"

So going off that, "diskutil listFilesystems" output, and the man page for diskutil (for roles list), something like the following might get the job done:

diskutil apfs addvolume diskN APFSX TimeMachine -quota NNGB -role T

NNGB is how many gigs you want to limit the volume/partition to. Quote: "GB is power of ten gigabytes where the multiplier is 1000000000 (1 x 10ˆ9)"

APFSX makes it case-sensitive.

Role of T is for Time Machine and found under "addVolume" in the man page.

diskN is the disk number for the APFS container returned from "diskutil list" command. In the following example, "disk3" is the APFS disk/container I currently have mounted (which happens to have a Time Machine volume).

/dev/disk3 (synthesized):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      APFS Container Scheme -                      +4.0 TB     disk3
                                 Physical Store disk2s2
   1:                APFS Volume ⁨TM01Misc⁩                460.0 GB   disk3s1
   2:                APFS Volume ⁨TM01Secure⁩              105.3 GB   disk3s2
   3:                APFS Volume ⁨TimeMachine01⁩           457.7 GB   disk3s3

It sounds like that would just limit the size of the Time Machine volume, which I don't think would help. I.e., I don't think a filled TM volume is going to perform better because there's an unfilled volume next to it.

In this particular case, I'll be using a QLC drive. These typically rely on 'QLC conversion' to create a virtual SLC cache (they write 1 bit instead of 4 bits to each QLC cell, to provide SLC-like cache performance). The problem is that this takes up a lot of space and, being budget drives, they typically don't over-provision much QLC for this purpose. As a consequence, they borrow from storage space to create this virtual SLC cache. This is fine when the drive is empty. But as the drive fills, the amount of QLC space they provision for cache can decrease markedly, which substantially impacts the amount of writes the drive can sustain before it reduces to QLC speed. For more on this, check out these articles (they're in German but, if you open them in Chrome, Google will translate them for you): https://www.computerbase.de/thema/s..._der_unterschied_zwischen_slc_mlc_tlc_und_qlc https://www.computerbase.de/2021-03/crucial-x6-portable-ssd-test/

To avoid this with your scheme, the SSD would need to alter its behavior: Instead of writing both data and cache to the TM volume, it would need to write data to the TM volume, and use the reserved volume for cache. That's now how SSDs are configured; it would require a specially-designed SSD.

And there may be other effects besides the above that cause drive performance to change with fill%.

 

[LeonPro]

I've been wondering the same thing. I've actually been using a Samsung 870 EVO 4TB for my Time Machine backup for 2 years and counting and I can report the following:

- DriveDX is now reporting a high number of errors on the SSD (possibly because this drive is always being filled up and backed up every hour, every day)

- Spinning beach ball whenever Time Machine is running (not sure if it's related to the filled up SSD drive or an OS Monterey issue). I shouldn't have any slowdown issues with this Mac Pro 2019 just for Time Machine.

Looking to see how we can properly limit the storage space being filled up so it isn't pushed to max

 

[Fishrrman]

A "Fishrrman broken record reply"...

If you want a backup routine that DOESN'T keep eating up your drive space with ridiculous "backups over backups over more backups"...

Switch to either CarbonCopyCloner or SuperDuper.
Either will create a cloned backup that does not continually "grow and grow" on you.

 

[LeonPro]

A "Fishrrman broken record reply"...

If you want a backup routine that DOESN'T keep eating up your drive space with ridiculous "backups over backups over more backups"...

Switch to either CarbonCopyCloner or SuperDuper.
Either will create a cloned backup that does not continually "grow and grow" on you.

I do have CCC which backs up my other drives at night. And Time Machine for my OS drive. I guess it's time to drop Time Machine and use CCC for OS to avert the SSD drive from filling up.

 

[saudor]

If it's a samsung drive, apparently, it uses unpartitioned space for wear levelling and general SSD maintenance but there are caveats in macOS. the problem with macOS is that it does not perform TRIM (unmap) on USB-based drives. Windows will trim a USB SSD provided the enclosure supports UASP protocol and tells the SSD controller which blocks can be purged.

Even if you could get Time Machine to prune at 75%, those deleted files arent marked as unused at the SSD block level. So the wear levelling likely won't happen as efficiently, which leads to premature death. There are some evidence of TRIM happening on certain enclosures but still not 100% sure if it's at the file system or block level.

 

[ignatius345]

I'm planning to use an a 4 TB external SSD for Time Machine backups (that way accessing TM won't be excessively slow, like it is with my HDD). To maintain performance, you don't want an SSD to fill up, but that's eventually what TM will do—keep adding snapshots until there's space for no more:

View attachment 2047286

There are various recommendations for the max usage on SSD's; Anand Shimpli gave 75% filled, so let's use that. Is there a way to get TM to start pruning backups when it gets to that point?

The alternative would be to prune the backups manually, but it would be tedious to do that regularly once the drive gets to 75%.

How responsive do you really need your Time Machine drive to be? Personally, I find it more useful to have more backups (and versions of files), even if it means the drive is fuller and, I guess, a little less responsive. It's still going to be much faster than your HDD. Besides, I suspect the drive interface is going to be more of a bottleneck than anything else.

 

[theorist9]

How responsive do you really need your Time Machine drive to be? Personally, I find it more useful to have more backups (and versions of files), even if it means the drive is fuller and, I guess, a little less responsive. It's still going to be much faster than your HDD. Besides, I suspect the drive interface is going to be more of a bottleneck than anything else.

So that's a good question, since I don't know what the point of diminishing returns would be with successively better backup peformance, not having been able to test it myself. Really, it's part of a bigger question involving my general locally-attached backup solution, which utilizes a combination of Time Machine (for versioning) and Carbon Copy Cloner (for backup). My remote archival backup is CCC only—if I lose everything in my home, and have to go to my remote backup, I can live without the TM history.

And what I've found, at least with my HDD, is that I need a lot more performance from my backup drives, partly to get quicker interaction when I do "Enter Time Machine" (which, granted, I don't do much), and partly to avoid the current problem that, when CCC and TM are operating together, the computer becomes noticeably laggy (I can usually run the backup overnight, but not always). Plus, with CCC, if I use CCC's "Find and Replace Corrupted Files" option, the incremental backup takes several hours, and my iMac has problems waking after this. So the TM performance is just a part of this. In addition, since I'm running Monterey, and encrypt all my backups, I can't use HFS anymore, and APFS doesn't work as well on HDDs.

Thus I've decided to switch to SSDs in the hope that having to take only minutes instead of hours for backup operations will obviate all those issues. And most who've switched to SSDs for their backups say it makes for a smoother experience.

But since I've not yet tested this, I don't know what the point of diminishing returns would be. The cheapest and simplest option to have 4 TB for TM and 2 TB for CCC (that would be enough for me) would be to attach one 4 TB Kingston XS2000 ($400) to one 5 Gb/s USB port for the TM, and another 2 TB portable SSD (maybe the Kingston, maybe something else, ~$200) to another open 5 Gb/s port, and be done. [I can't directly attach either to TB ports without a dock, since those are both take by external monitors. For more money, I could get a dual NVMe Thunderbolt dock, and outfit it with in a 4 TB Crucial P3 ($290) and a 2 TB Solidigm P41 plus ($170) (more than the first solution, even though the drives are, cheaper b/c I'd have to buy dual-NVME dock) (I'd then attach the monitor to the dock). That would give me about 2-3 x the speed, but would it make a difference in practice? Without testing both myself, IDK.

 

[gilby101]

As I tried to explain in my starting post, my specific issue with TM's slow speed (and my main reason for getting an SSD for TM) occurs when accessing TM (i.e. entering TM and scrolling through the snapshots).

I never access my TM drive using that interface. No performance issues using Finder to access backups.

HDD is fine for Time Machine once the first backup is complete. Just make sure you get a non-SMR HDD (most cheap small HDD are SMR).

And use BackupLoupe https://www.soma-zone.com/BackupLoupe/ to 1) create and maintain an index of your backups and 2) pinpoint size issues.

 

[gilby101]

Switch to either CarbonCopyCloner or SuperDuper.
Either will create a cloned backup that does not continually "grow and grow" on you.

Sorry, that is a different sort of backup. If it doesn't grow, it is not keeping a history of past changes.

 

[theorist9]

A "Fishrrman broken record reply"...

If you want a backup routine that DOESN'T keep eating up your drive space with ridiculous "backups over backups over more backups"...

Switch to either CarbonCopyCloner or SuperDuper.
Either will create a cloned backup that does not continually "grow and grow" on you.

Sorry, that is a different sort of backup. If it doesn't grow, it is not keeping a history of past changes.

I never access my TM drive using that interface. No performance issues using Finder to access backups.

So I could use CCC to do some form of versioning. Essentially, you tell it to save APFS snapshots, which are the same thing TM saves. Plus with CCC, you can tell it to stop saving stuff when free space goes below some fixed amount you specify:

Leveraging Snapshots on APFS Volumes - Bombich Software

bombich.com

Now that might appear to be a solution. But there is a key problem with this, because it requres using Finder to search the snapshots.

With TM if you need to look back through older versions of of a given file to find the one you want, you just go to the folder with the file you want, and as you search backwards you can scan back through just that folder, which allows you to easily open & check the file until you find the version you want. By contrast, if you use Finder, it's far more tedious. For each snapshot you want to access, you need to open that snapshot, drill down to find the relevant folder, and open the file.

The point is the TM interface is far more functional for doing this than using Finder—because that's precisely what it's designed for.

 

[theorist9]

I've been wondering the same thing. I've actually been using a Samsung 870 EVO 4TB for my Time Machine backup for 2 years and counting and I can report the following:

- DriveDX is now reporting a high number of errors on the SSD (possibly because this drive is always being filled up and backed up every hour, every day)

- Spinning beach ball whenever Time Machine is running (not sure if it's related to the filled up SSD drive or an OS Monterey issue). I shouldn't have any slowdown issues with this Mac Pro 2019 just for Time Machine.

Looking to see how we can properly limit the storage space being filled up so it isn't pushed to max

At first I thought of recommending you try a one-time prune of TM down to, say, 70% capacity, just to do the experiment to see if that improves the behavior. However, Apple doesn't recommend pruning -- instead they say:

"If you do run out of space, it’s best to connect a new backup disk." [https://support.apple.com/en-gb/guide/mac-help/mh15137/mac]

I think the reason is that each snapshot depends on the ones before it, so if you simply delete snapshots you'll mess up those dependencies. [Having said that, I think if you pruned backwards from the most recent snapshots, you'd be OK--but I'm not sure.]

So really, the only way to test if your issues are from the drive being full would be to copy your SSD's contents over to another drive* (say an HDD, which would be good for archival purposes), erase and reformat your SSD, and start TM from scratch.

*You'll want to check that TM works on the new drive before you erase your SSD, since copying TM to a new drive can be tricky (https://discussions.apple.com/thread/253674779)

 

[LeonPro]

^Thanks for thoughts. I today eneded up removing Time Machine feature and then reformatting the same SSD drive as a CCC clone.

I realise I have never used Time Machine's feature to actually, umm, go back in time. I just need a backup should my main OS drive goes down.

 

[theorist9]

I realise I have never used Time Machine's feature to actually, umm, go back in time.