Where has my HD space gone?

[myotis]

I wonder if any one can help resolve this. Mid 2011 11" air with 128Mb HD 4gb RAM.

I have had a recurring disappearance of hard drive space down to zero on occasions. I can release some space and watch vanish release some more and watch it vanish. But most of the time it will settle for a while at about 3-4gb.

The last couple of days I have been able to watch it get eaten up, with the kernal_task in Activity Monitor recording files being written, and me watching the available free space drop at the same rate. About a Gb a second.

I released (deleting files) 7Gb which dropped to zero, a reboot brought it back 3.9gb. I then released 5.5gb, and watched it drop to 1.4gb, and a reboot brought it back to 1.9gb.

I have now reset NVRAM and SMC, and released another 10gb of space, which is still dropping but much more slowly while I wrote this it dropped from 12.2gb to 10.8gb.

But I am really puzzled by the different reports of available space.

Using Pathfinder:

The space on my MacBook air is 41.6gb available with 110.4gb used
The available space on the Macintosh Air HD is 10.6gb with 109.4gb used.

Daisy disk also reports 109.4gb available but with 80.86 used so there is 28gb unaccounted for.

And in the last few hours I have deleted over 25gb worth of files, and over the last few days probably a few more Gb.

Assuming the NVRAM and SMC has solved the issue , where might I look for my missing hard drive space.

HD space is now at 9.5gb.

I realise there are probably hundreds of possible answers to this, but I would appreciate any suggestions, as its seems I potentially have a fair amount of free space available if I could only find it.

Thanks,
Graham

 

[organicCPU]

caches, sleepimage, swapfile
You seem to work really hard on your internal 128 GB drive. What about upgrading it, e.g. with an OWC Aura (Pro) 6G? If that's no option, I recommend using an external drive. For maximum speed consider a SSD drive connected over TB. For data that does not needed to be accessed often a HDD over USB 2.0 is just fine.

 

[Weaselboy]

Do you have Time Machine turned on? That stores some local snapshot files in a hidden folder and can use a lot of space.

Run this command in Terminal and tell us what it says. It will show how much space is being sued for local snapshots.

sudo du -hs /.MobileBackups

 

[myotis]

Thanks, i have used Clean my Mac to empty caches, and the sleepimage, swapfile0 and swapfile1 are 1.1.gb each. Any idea why I have two swap files?

But I am more concerned about the space I cannot find/ identify ie the 41.6gb and the 28gb.

Yes, I do have a problem with the SSD size and almost immediately regretted not getting the 256gb version. I confess I thought that the SSD couldn't be upgraded on the Macbook Air so I will certainly look into that. Hopefully its got a few years left in it yet.

Cheers,

Graham
[doublepost=1485976042][/doublepost]

Do you have Time Machine turned on? That stores some local snapshot files in a hidden folder and can use a lot of space.

Thanks, I'm pretty confident I don't.

output from command

du: /.MobileBackups: No such file or directory

Which confirms I don't.

Cheers,
Graham

 

[organicCPU]

But I am more concerned about the space I cannot find/ identify ie the 41.6gb and the 28gb.

Maybe this can get you an idea:
http://grandperspectiv.sourceforge.net

 

[myotis]

Maybe this can get you an idea:
http://grandperspectiv.sourceforge.net

Thanks, I've downloaded this but not sure what its showing me that DaisyDisk isn't showing me.

Is there something clever that I am missing where grandperspective will show me files that are hidden to DaisyDisk.

it does however report 17.6gb of miscellaneous used space but as in my OP I still have 28Gb not accounted for in Daisy disk.

I am still puzzled that I have deleted about 20 - 25Gb of files in the last couple of days and that space has just vanished , and although I now have 10gb free, and its not being eaten away now, this is me just back to where I was a few days ago. So in spite of deleting these file, I haven't gained any space.

Cheers,

Graham

 

[Weaselboy]

https://daisydiskapp.com/manual/en.lproj/Topics/AdminScan.html

Daisy Disk is actually better because it can "scan as admin" to show system files other apps like that don't show. Try turning on that setting and scan.

 

[myotis]

https://daisydiskapp.com/manual/en.lproj/Topics/AdminScan.html

Daisy Disk is actually better because it can "scan as admin" to show system files other apps like that don't show. Try turning on that setting and scan.

Well that was little more complicated than expected, as I had the app store version, and needed to convert to the stand alone version to use the admin feature.

It seems the big bit of space (24.4gb) being used is in the system library caches com.apple.coresymbolicationd.

with 5 files called

data
grow.z5UtRNm

etc

There is 1gb for spotlight index and 2.3gb hidden space (which I can't open)

Is this a reasonable size for these caches? Is this something I can delete?

Cheers,

Graham

 

[organicCPU]

Is this a reasonable size for these caches? Is this something I can delete?

Usually caches can be deleted, but they'll be rebuilt to speed up your Mac. Here on stackexchange is a bit more on com.apple.coresymbolicationd.
As the new OSs are getting more complex, one should have enough free space on the system drive to let the OS doing its job and not worrying too much about each GB of unknown files. I understand, that for your drive size that's not so easy to accept. Just this morning I swapped out a 500 GB with a 1 TB drive, because I couldn't release any more space in a sensible way.

Any idea why I have two swap files?

There are as many swapfiles as needed, not just two. As soon as you restart your Mac, they'll get deleted and rebuilt.

Thanks, I've downloaded this but not sure what its showing me that DaisyDisk isn't showing me.

I thought that a free tool would be better than none, but as you already have Daisy Disk, there is no need for Grand Perspective.

 

[myotis]

Usually caches can be deleted, but they'll be rebuilt to speed up your Mac. Here on stackexchange is a bit more on com.apple.coresymbolicationd.
As the new OSs are getting more complex, one should have enough free space on the system drive to let the OS doing its job and not worrying too much about each GB of unknown files. I understand, that for your drive size that's not so easy to accept. Just this morning I swapped out a 500 GB with a 1 TB drive, because I couldn't release any more space in a sensible way.

There are as many swapfiles as needed, not just two. As soon as you restart your Mac, they'll get deleted and rebuilt.

I thought that a free tool would be better than none, but as you already have Daisy Disk, there is no need for Grand Perspective.

Now deleted using

sudo rm -rf /System/Library/Caches/com.apple.coresymbolicationd

And I now have 36.1gb free.

My Mac Mini has a 500Gb SSD + 750Gb HD + 2Tb external hard drive, but the Macbook Air was bought as a low cost easily transported option, however, as I said I have struggled with the HD size from the beginning.

As I quoted the numbers from DaisyDisk in my OP, I just assumed that Grand Perspective was going to give me some additional information, buts as it so happens I have now made Daisy Disk more useful than it was.

My new problem is the hidden space which was 2.4gb and is now 3.6Gb, I have tried running repair permissions and there is no fault found, but I can't access this hidden space.

Thanks for your help.

Cheers,

Graham

 

[organicCPU]

Congratulations that you managed to free up so much space.

My new problem is the hidden space which was 2.4gb and is now 3.6Gb, I have tried running repair permissions and there is no fault found, but I can't access this hidden space.

You could try to boot your MacBook Air in Target Disk Mode and scan the drive from the Mac mini.
Depending on the OS you could also try to disable SIP and/or enable the root user to perform the scan again.
However, there is always a risk that something goes wrong, especially when performing tasks like this, means backing up before can't be wrong...

 

[Weaselboy]

My new problem is the hidden space which was 2.4gb and is now 3.6Gb

Try this command in Terminal that will show everything including hidden folders.

sudo du -d 1 -x -c -g /

 

[myotis]

Congratulations that you managed to free up so much space.

You could try to boot your MacBook Air in Target Disk Mode and scan the drive from the Mac mini.
Depending on the OS you could also try to disable SIP and/or enable the root user to perform the scan again.
However, there is always a risk that something goes wrong, especially when performing tasks like this, means backing up before can't be wrong...

Try this command in Terminal that will show everything including hidden folders.

sudo du -d 1 -x -c -g /

My list of folders is:



1 /.DocumentRevisions-V100

1 /.fseventsd

1 /.Spotlight-V100

0 /.Trashes

0 /.vol

15 /Applications

1 /bin

0 /cores

1 /dev

1 /home

1 /Incompatible Software

8 /Library

1 /net

0 /Network

1 /opt

1 /PhotoTemp

5 /private

1 /sbin

5 /System

35 /Users

10 /usr

1 /Volumes

78 /

78 total

But not sure what this tells me, as I think these were all found by DaisyDisk. When I run repair permissions it hangs for minutes on index.html.

I will have a look at the Target mode, but of course I could be going down a false path, as its the DaisyDisk manual that suggests the inability to open this hidden space folder is a permissions error. Verifying permissions with disk utility and clean my mac, suggests I don't have a permissions problem.

Thanks,

Graham

 

[organicCPU]

Verifying permissions with disk utility and clean my mac, suggests I don't have a permissions problem.

If you can't narrow down with Daisy Disk where those hidden files are located, a good approach would be to outperform permissions as said in my last post.
If Daisy Disk is right and there are some uncommon permissions, keep in mind that Disk Utility, and probably Clean my Mac relies on Disk Utility for that task, are just checking and restoring original permissions related to the Mac OS X system and Apple apps. They don't check other permissions that are i.e. set by third party installers nor do they check or repair the permissions of your home folder.

 

[myotis]

If you can't narrow down with Daisy Disk where those hidden files are located, a good approach would be to outperform permissions as said in my last post.
If Daisy Disk is right and there are some uncommon permissions, keep in mind that Disk Utility, and probably Clean my Mac relies on Disk Utility for that task, are just checking and restoring original permissions related to the Mac OS X system and Apple apps. They don't check other permissions that are i.e. set by third party installers nor do they check or repair the permissions of your home folder.

Yes, I kind of assumed that Clean my Mac was probably just running the Mac disk utility. i might give the home folders permission repair that you linked to a go.

When you say "outperform" (not a term I have ever heard) do you just mean the advice about using the macbook air as an external drive on my Mini so I can run the disk repair utility.

Thanks,
Graham

 

[organicCPU]

When you say "outperform" (not a term I have ever heard) do you just mean the advice about using the macbook air as an external drive on my Mini so I can run the disk repair utility.

To boot as a Target Disk can circumvent the permissions on the drive that doesn't allow you to find what you're searching for. You can then scan with Daisy Disk for the hidden files, that shouldn't be blocked by permissions anymore. That's probably the easiest way to find out where the files are.
However, I'm not sure if that will help you to free more space, as it just points you to the files you want to discover. If you should touch these hidden files or not is another question.
Repairing permissions in Target Disk Mode should have no benefits over a repair while regular booted. To reset home folder permissions is not really necessary and probably useless to find the hidden files. I just wanted to explain that the repairing permissions function doesn't reset permissions on the whole drive.

 

[Weaselboy]

78 /

78 total

I'm not seeing anything unusual here. Is the disappearing space issue gone? If so, I'm not sure what we are trying to fix any longer?

I would not use Clean my Mac for anything and I would delete it. It can cause more problems than if fixes.

 

[myotis]

To boot as a Target Disk can circumvent the permissions on the drive that doesn't allow you to find what you're searching for. You can then scan with Daisy Disk for the hidden files, that shouldn't be blocked by permissions anymore. That's probably the easiest way to find out where the files are.
However, I'm not sure if that will help you to free more space, as it just points you to the files you want to discover. If you should touch these hidden files or not is another question.
Repairing permissions in Target Disk Mode should have no benefits over a repair while regular booted. To reset home folder permissions is not really necessary and probably useless to find the hidden files. I just wanted to explain that the repairing permissions function doesn't reset permissions on the whole drive.

I'm not seeing anything unusual here. Is the disappearing space issue gone? If so, I'm not sure what we are trying to fix any longer?

I would not use Clean my Mac for anything and I would delete it. It can cause more problems than if fixes.

@organicCPU
Ok, I will maybe just leave well alone as everything is running a lot more smoothly now and the disappearing space problem seems fixed.

@Weaselboy

I was left trying to solve the "hidden space" reported by Daisydisk. You should be able to click on the "hidden space" label to see what is in it ( about 3Gb), but the DaisyDisk manual says if clicking on it doesn't do anything, it is probably a permissions issue, and this is what was happening.

I am naturally wary of programs like Clean My Mac, but before starting to use it several years ago, I was encouraged by the number of techy writers who said this was the only program of this type they would use. I have three Macs that I run it regularly on, and its useful to free up space and uninstall apps. I'm not sure how you identify when its causing problems. I'm not aware of any issues or problems until this one.

Is there something else you would suggest for general housekeeping, uninstalling, etc

Thanks,
Graham

 

[organicCPU]

Good to hear that everything is working again for you!

Is there something else you would suggest for general housekeeping, uninstalling, etc

I was thinking quite some time, if I would help you in answering that. I'll try.

Most, if not all, of these tools have big potential to mess up more than they'll help you in the long run. I can't recommend any third party maintaining tool for regular or daily use as there is usually no need for them. Are they all bad? Well, some give you a nice GUI for tasks you can perform with Terminal commands and some have additional functionality. So you might personally benefit, if you'll use them with caution and understand what exactly they're doing.

Most tools you'll ever need are packed into MacOS. Some have a nice GUI, some have to be accessed through Terminal.
Is there really no voodoo? Well, yes and no! The voodoo is your experience and how you manage to fix problems or prevent that they'll ever occur. Do I use some extra tools? Yes, I do! Why? Either I need a well proved solution for the task or I'm just too lazy to use the Terminal or like the nice GUI.

I use tools for various things like backup (Carbon Copy Cloner), occasionally DYLD and font cache cleaning (OnyX, FontNuke), occasionally finding duplicates (dupeGuru), finding files (EasyFind), occasionally recovering files from friends (ddrescue). The only one of these tools I really don't want to miss is my backup tool, the others I could give a miss. I'm always curios in testing various other tools, but don't really need them. That said, I don't recommend you to install any of the mentioned apps until you really think you benefit from them.

Maybe your set of tools will look totally different, because your demand is completely different and that's entirely no problem then. If you really think you need Clean My Mac, like it, know to use it and it really helps you, then you should continue using it and don't care what others say. It's really no problem that I wouldn't use it and personally agree with @Weaselboy about that. Nevertheless, I think your Mac is there to get things done in the first place and not to maintain it. So just hve fun using it...

 

[myotis]

I was thinking quite some time, if I would help you in answering that. I'll try.

If you really think you need Clean My Mac, like it, know to use it and it really helps you, then you should continue using it and don't care what others say.

Thanks for the effort you have put into your answer.

I don't actually know if I "..need Clean My Mac"

But I am usually struggling with Disk Space and Clean My Mac (however it does it) would recover disk space that shutting programs down or restarting doesn't. I can recover, what seems to be non-trivial amounts from my Mac Mini, e.g 50-100gb by running Clean my Mac.

Equally, uninstalling programs by dragging to the trash (assuming no uninstaller program) would normally leave bits and pieces behind, that Clean My Mac would remove. I install and uninstall a lot of programs, that I have a temporary need to have access to, and I feel happier if I can get rid of the remnants.

And its a convenient way to run a check on disk permissions etc, which I had assumed was probably just running the Mac utilities.

So its primarily just convenience, and the good reviews that led me to Clean my Mac, when compared to the generally negative reviews of all the other similar programs.

However, you have both given me food for thought, and maybe I need to rethink this. Although, I put a lot of thought into it at one time, I have given it no thought at all for several years now.

Cheers,
Graham

 

[Oleg K.]

@myotis, 2-3 GB of "hidden space" is about normal for a start-up disk, so I'd not be worried about it. When the "hidden space" becomes about that small, like 2-3GB, DaisyDisk may even omit it altogether from view. It's assumed that the space is used by the files system's own overhead.

When the "hidden space" becomes like 5-7 GB or more, you're right that usually fixing the permissions helps. The messed up permissions are a "feature" of the HFS file system In older macOS versions there were even the special buttons in Disk Utility titled "Verify Permissions" and "Repair Permissions" for that. Later, I think starting from El Capitan, they merged all 4 buttons into one button "First Aid".

If the Disk Utility doesn't help, it becomes more tricky. I myself had a problem like that. No amount of research could reveal where like 10-15 GB of hidden space reported by DaisyDisk were going. I delved really deep and it turned out in the end that my HFS catalog file (an internal structure of HFS) had some issues, and I had to use the utility fsck_hfs to do a procedure called "rebuild catalog file". Also, AFAIR, it gave me an error and so I had to reboot in recovery mode in order for this procedure to complete successfully. After it succeeded, the "hidden space" was gone!

(WARNING! That procedure is not safe! If you want to try rebuilding the catalog file, make sure you've got an up-to-date Time Machine backup of your entire drive! Sometimes the command may fail and leave your disk broken, so that only reformat will help, or restore from the backup. Do it on your own risk only when really necessary!)

 

[myotis]

@myotis, 2-3 GB of "hidden space" is about normal for a start-up disk, so I'd not be worried about it. When the "hidden space" becomes about that small, like 2-3GB, DaisyDisk may even omit it altogether from view. It's assumed that the space is used by the files system's own overhead.

Thanks, that's good to know, and yes it is just "first aid" now in el capitan.

Fortunately, i don't need to rebuild the catalogue - yet.

Cheers,

Graham