not understanding storage numbers

[Doug Lass]

There are a number of ways to assess how much information is on a storage device. I'm bothered by the fact that these numbers are kind of inconsistent. Can someone help me out? Here are numbers that correspond to the same time on my Mac (Monterey, silicon iMac).

*About this Mac/Storage says
Applications 21.48 GB
Documents 43.89 GB
OS 15.75 GB
Sys data 109.68 GB
--- Total 190.80GB

*Finder says
Applications 23.27 GB (22.26 GB on disk)
Library 8.43 GB (6.61 GB on disk)
System 28.36 GB (16.69 GB on disk)
User 61.93 GB (63.36 GB on disk)
--- Total 121.99 GB 107.92GB on disk

*Disk utility says
name of disk 15.75 GB
data 174.71 GB
--- Total 190.46 GB

*Bootable Backup disk reports
136.7 GB on disk

These numbers aren't very consistent. "Data" is confused. "System" is evidently not the same as "OS". Even "Applications" are little squirrely. Total listed by Finder is much lower than what About this Mac/Storage or Disk utility says. To the extent my disk really has 190 GB in it, I'm missing some 80 GB of stuff that is somehow NOT Applications, Library, System, or User. What is it?? Unless my Bootable Backup is somehow compressed, it sure isn't reflecting what is supposed to be on my system disk. Also, what's this business about "on disk" versus, presumably, not "on disk" when I'm polling info about info on a disk?

Would very much appreciate some guidance here. Where can I get a REAL COMPLETE UNAMBIGUOUS listing of what is on my disk? Or at least some written explanation of what all these terms really mean.

 

[IngoX]

Howard Oakley gives some explanations on his blog:

Explainer: Disk free space

Confused by this on APFS? So you should be, as it’s complicated by features like snapshots, sparse files, and the file system itself. There’s no single figure.

eclecticlight.co

Quantum computing and APFS: free and used space

Why measuring free and used disk space is not so simple in APFS, and which estimates you should trust.

eclecticlight.co

 

[Doug Lass]

That's interesting. Thank you. But I'm interested in specifics here. I'm missing some 80 GB of stuff that is somehow NOT Applications, Library, System, or User. Where is it?

I'm beginning to think that it's in Time Machine backups, which are in "hidden" directories. Now, I've just migrated my Time Machine backups to an external disk, and I'm trying to understand how to purge all the backups in my system disk. Not obvious (Monterey).

 

[IngoX]

With Monterey you can inspect, mount and delete TimeMachine snapshots in Disk Utility.

But I’m nowadays content with letting TimeMachine and macOS do its thing. Apple hasn’t publicly documented everything about APFS and it’s still a work in progress. Space usually reappear whith time…

Disk Utility now has full features for managing snapshots

New for Monterey: Disk Utility now offers complete features for the management and maintenance of snapshots.

eclecticlight.co

 

[IngoX]

If you start digging into your snapshots, make sure to show the column ”Private Size”

Understanding snapshot data in Disk Utility

Explains the identifiers, sizes and tidemark data shown for snapshots in Disk Utility, in Monterey.

eclecticlight.co

 

[Doug Lass]

Oh wow, that's slick! Purging Time Machine backups just got a lot easier. Thank you.

Though when I do that, and purge four such backups, the reported amount of space used in that volume by Disk Utility doesn't change much.

I am left with
com.apple.asr.2591 Nov 8, 2021 at 9:14 AM 245.1 GB (at block 59,839,501) 45.86 GB (45,858,930,688 bytes) Snapshot
com.apple.asr.45819 Jan 29, 2022 at 5:40 PM 245.11 GB (at block 59,840,607) * 56.1 GB (56,099,831,808 bytes) Snapshot

What are those, and now that I've migrated my Time Machine backups to an external disk, do I need them anymore? Are they what's taking up all the extra storage space? I believe .asr files have something to do with system restoration.

 

[BigBlur]

That's interesting. Thank you. But I'm interested in specifics here. I'm missing some 80 GB of stuff that is somehow NOT Applications, Library, System, or User. Where is it?

Those are just the 4 main folders that are visible when you view "Macintosh HD". There are quite a few hidden files and folders at that level you aren't accounting for.

Press '⇧⌘.' (Shift + Command + .) in Finder to show/hide those hidden files and folders.

 

[Doug Lass]

Well, I keep erasing these in Disk Utility on my system disk,and they keep coming back!!!

com.apple.TimeMachine.2022-01-31-094411.local Today at 9:44 AM 245.08 GB (at block 59,835,101) 56.58 GB (56,577,765,376 bytes) Time Machine Snapshot

Now, I have SPECIFICALLY identified an external disk for my Time Machine backups. Why in the world is Time Machine putting stuff here? Time Machine shouldn't be doing anything anymore on my system disk.

 

[BigBlur]

Well, I keep erasing these in Disk Utility on my system disk,and they keep coming back!!!

com.apple.TimeMachine.2022-01-31-094411.local Today at 9:44 AM 245.08 GB (at block 59,835,101) 56.58 GB (56,577,765,376 bytes) Time Machine Snapshot

Now, I have SPECIFICALLY identified an external disk for my Time Machine backups. Why in the world is Time Machine putting stuff here? Time Machine shouldn't be doing anything anymore on my system disk.

This is normal. Time Machine keeps local snapshots on the system. You shouldn't have to delete these yourself. MacOS will delete them automatically as they age or when more space is needed for other things.

About Time Machine local snapshots - Apple Support

Time Machine lets you restore files from local snapshots of the files on your Mac, even when your Time Machine backup disk isn't available.

support.apple.com

 

[Doug Lass]

Well, what's the point of telling Time Machine to back up to an external disk if it's going to keep throwing stuff on my system disk??

 

[Doug Lass]

Those are just the 4 main folders that are visible when you view "Macintosh HD". There are quite a few hidden files and folders at that level you aren't accounting for.

Press '⇧⌘.' (Shift + Command + .) in Finder to show/hide those hidden files and folder

OK, I did that.I add up the sizes of ALL FILES that are presented, and the total comes out to be 126.74 GB for"Macintosh HD". But Finder reports for that disk Used: 190.1 GB (190.1 GB on disk).

Something doesn't add up here. Not by a long shot.

 

[BigBlur]

Well, what's the point of telling Time Machine to back up to an external disk if it's going to keep throwing stuff on my system disk??

This is just how Time Machine works...it creates snapshots every hour and stores them locally. It then copies them to the Time Machine drive on a schedule or when it has access to it. Not everyone is going to have their Time Machine drive connected 24/7 so that it can copy over the hourly snapshots. It also leaves the latest ones on your machine so that you can still restore files when you don't have access to your Time Machine drive.

As for the file size discrepancies, the file system is a very complex thing, especially now with APFS. For example, I could have a 200MB file on my desktop. If I make a copy of that file, my total disk space used is actually still 200MB even though both files themselves will report as being 200MB each. The two files are basically just pointing to the same bits on the drive. Now if I make a 5MB change to that file, my total disk space used is around 205MB, as the copy just keeps track of the delta changes. The snapshots work in a similar way, the bits aren't copied to another part of the drive and taking up more space. They are pointing to the actual bits and keeping track of the delta changes. This probably explains it better than I can.

Because of this, I don't think you will ever get your numbers to jive by checking the file size of every file and folder in Finder. Are you just trying to understand how this all works? Or are you actually trying to solve some other problem by freeing up some disk space?

 

[u_int16]

The application “disk-inventory-x” is free, and should map every single file.

Consider using that, and letting us know where the ghost storage is! Maybe run a first aid on your volumes, for good measure.

 

[Doug Lass]

This is just how Time Machine works...it creates snapshots every hour and stores them locally. It then copies them to the Time Machine drive on a schedule or when it has access to it. Not everyone is going to have their Time Machine drive connected 24/7 so that it can copy over the hourly snapshots. It also leaves the latest ones on your machine so that you can still restore files when you don't have access to your Time Machine drive.

As for the file size discrepancies, the file system is a very complex thing, especially now with APFS. For example, I could have a 200MB file on my desktop. If I make a copy of that file, my total disk space used is actually still 200MB even though both files themselves will report as being 200MB each. The two files are basically just pointing to the same bits on the drive. Now if I make a 5MB change to that file, my total disk space used is around 205MB, as the copy just keeps track of the delta changes. The snapshots work in a similar way, the bits aren't copied to another part of the drive and taking up more space. They are pointing to the actual bits and keeping track of the delta changes. This probably explains it better than I can.

Because of this, I don't think you will ever get your numbers to jive by checking the file size of every file and folder in Finder. Are you just trying to understand how this all works? Or are you actually trying to solve some other problem by freeing up some disk space?

Well, OK. It does seem strange that if Time Machine offers to put its backup on another disk, it STILL chooses to put it locally. I mean, if my external Time Machine drive isn't going to be connected 24/7 (mine is), I'll simply DECIDE to keep it all local. But Apple isn't letting me do that. It's treating my eternal drive as an "additional" Time Machine storage location. It just seems like a really odd way to do it. If you want to let me choose where to put the backup, then let me do it.

Now, I don't have identical copies of lots of stuff on my drive, but what I'm learning is that the available drive space is actually pretty fluid, in that Time Machine files, and maybe Spotlight files, are pretty expendable. So if they're taking up a large piece of the drive, that space is completely usable. They just get deleted if the space is needed. If that's the case, it would be nice to have a measure of what is expendable, and separately, what is not expendable. What is not expendable is the System, Applications, Library, and my data (User). I guess the latter is the sum of those four pieces I quoted above. The 190GB above includes all the expendable stuff. Is that correct?

I guess what I'm trying to do is (1) figure out what all these numbers really mean and (2) economize on drive usage. I have a 256 GB internal SSD, and I'm wondering how much non-expendable stuff it'll allow me to have.

 

[Doug Lass]

This is getting off topic, but with regard to local versus remote Time Machine backups, I see this

How to Free Up Space Used By Time Machine's Local Backups on Your Mac

Time Machine doesn't just back up to external drives.

www.howtogeek.com

That's PRECISELY what I want to do. To prevent local backups. They say it's simple. You just do sudo tmutil disablelocal in Terminal! But when I do that in Monterey, I just get an error message back disablelocal: Unrecognized verb.

So, someone, anyone, how do you really disable local Time Machine backups??

 

[clevins]

You DO NOT WANT to turn off snapshots for the reasons above.

I have a higher level question though- - why do you care about this? If you're running low on disk, that's one good reason or if you just are curious, that's cool too. But with modern OSes it's almost never a good idea to start manually micromanaging things like how they do backups unless you really really need to and know exactly what you're doing and the side-effects of doing the manual overrides.

 

[SecuritySteve]

This is getting off topic, but with regard to local versus remote Time Machine backups, I see this

How to Free Up Space Used By Time Machine's Local Backups on Your Mac

Time Machine doesn't just back up to external drives.

www.howtogeek.com

That's PRECISELY what I want to do. To prevent local backups. They say it's simple. You just do sudo tmutil disablelocal in Terminal! But when I do that in Monterey, I just get an error message back disablelocal: Unrecognized verb.

So, someone, anyone, how do you really disable local Time Machine backups??

The disable/enable local backup feature was removed at some point between now and High Sierra when this guide was written. You cannot disable this feature anymore. If you do not like how Time Machine works, I would suggest looking into alternatives.

Here's one alternative I've noticed other users are fond of: Carbon Copy Cloner

Carbon Copy Cloner | macOS Backup Software | Bombich Software

bombich.com

 

[Doug Lass]

You DO NOT WANT to turn off snapshots for the reasons above.

I have a higher level question though- - why do you care about this? If you're running low on disk, that's one good reason or if you just are curious, that's cool too. But with modern OSes it's almost never a good idea to start manually micromanaging things like how they do backups unless you really really need to and know exactly what you're doing and the side-effects of doing the manual overrides.

I don't want to turn off Time Machine snapshots. I'd like to restrict them to an external disk. I saw no reasons propounded above that would discourage me from doing that.

But let's talk about why. I completely understand that Time Machine files are "expendable", in that they don't really restrict the content you save. As you need more room, the older ones just get automatically trashed. But, as a result, you've got a backup disk with variable size, and it isn't always clear what you've managed to save. For that reason, I'd like to have a DEDICATED Time Machine backup disk that will always be connected. Seems like a simple proposition, no? But it is just nuts that Apple won't let you do that. No question that Time Machine backups take computing power and also are limited by disk speed. To the extent I wouldn't be doing disk sharing if I had a wholly external Time Machine drive, it might interfere somewhat less with other work.

 

[Doug Lass]

The disable/enable local backup feature was removed at some point between now and High Sierra when this guide was written. You cannot disable this feature anymore. If you do not like how Time Machine works, I would suggest looking into alternatives.

Ah, no wonder! I like how Time Machine works, but I just wanted to delegate it to one particular disk. Sounds like people are grossly offended by that idea, but it seems pretty simple to me. I guess Apple was grossly offended as well.

So I guess I'm stuck with Time Machine BOTH on my system disk and my external disk. Sounds like the right strategy is just to ignore the former.

As to bootable backups, I've been doing SuperDuper for decades, and Dave Nanian has been extremely helpful in getting it to work beautifully with modern operating systems. CCC is good,but I like the user interface on SuperDuper better. Bootable backups offer some flexibility that Time Machine does not, and vice versa. So I'd like to have both.

 

[SecuritySteve]

Ah, no wonder! I like how Time Machine works, but I just wanted to delegate it to one particular disk. Sounds like people are grossly offended by that idea, but it seems pretty simple to me. I guess Apple was grossly offended as well.

So I guess I'm stuck with Time Machine BOTH on my system disk and my external disk. Sounds like the right strategy is just to ignore the former.

As to bootable backups, I've been doing SuperDuper for decades, and Dave Nanian has been extremely helpful in getting it to work beautifully with modern operating systems. CCC is good,but I like the user interface on SuperDuper better. Bootable backups offer some flexibility that Time Machine does not, and vice versa. So I'd like to have both.

Keep in mind that you don't have full backups on your system disk if you are delegating to an external disk. Only the most recent snapshot is kept locally, so you still are saving a ton of storage by moving your backup to an external disk.

 

[Doug Lass]

That's a fair point, but I'm trying to insulate against total system disk failure. If my system disk goes south, I want to be able to recover. Of course, a bootable backup will do that as well, but Time Machine has a few benefits that bootable backups don't (and vice versa).

 

[clevins]

Ah, no wonder! I like how Time Machine works, but I just wanted to delegate it to one particular disk. Sounds like people are grossly offended by that idea, but it seems pretty simple to me. I guess Apple was grossly offended as well.

we're not offended, we just realize how it works. Time Machine cannot, in general, assume that the external drive will always be available, so it writes a snapshot locally in near-real time and then to the external drive periodically or when next available.

You're the one who, for some reason, is obsessing over this even though you admit it doesnt really matter.

 

[clevins]

That's a fair point, but I'm trying to insulate against total system disk failure. If my system disk goes south, I want to be able to recover. Of course, a bootable backup will do that as well, but Time Machine has a few benefits that bootable backups don't (and vice versa).

And you can. To the last snapshot on the external disk. Which should be at most about an hour old if it's always connected as you say it is above. So... at worst you lose an hour's work if it's not synced and your system disk completely dies with zero warning.

Since I use a MacBook, I also do near real time offsite backups (to Backblaze) which also guards against the complete loss of on-site backups.

 

[Doug Lass]

we're not offended, we just realize how it works. Time Machine cannot, in general, assume that the external drive will always be available, so it writes a snapshot locally in near-real time and then to the external drive periodically or when next available.

You're the one who, for some reason, is obsessing over this even though you admit it doesnt really matter.

To be honest, it's up to ME, not Time Machine, to decide whether the external disk is available. When I choose backup to the external disk, I am specifically obligating myself to that choice. If I don't make that specific choice, then I'm happy to let Time Machine take care of me.

I don't think I admitted it really didn't matter. I was just saying that if I have a known, and very large place to store snapshots, I have assurance that backups will be available for a very long time. Now, that's not wildly important, but I have an extra disk, so why not? Now, using words like "obsessing" aren't really polite or constructive. I'm asking a simple question, and I appreciate informative answers.

 

[Doug Lass]

And you can. To the last snapshot on the external disk. Which should be at most about an hour old if it's always connected as you say it is above. So... at worst you lose an hour's work if it's not synced and your system disk completely dies with zero warning.

Since I use a MacBook, I also do near real time offsite backups (to Backblaze) which also guards against the complete loss of on-site backups.

Um, if my internal system disk goes south, I'm going to lose a lot more than an hours work, even if I'm backed up by Time Machine to an external disk. Unless I get a speedy repair person to dismantle the machine and replace the internal disk. That's why a bootable backup is the way to go. In that case, I'm back in action in a minute or less.