Hard drive space being slowly eaten, then freed at restart

[timelord726]

Okay, guys. I've got a strange one for you.

My iMac is having its hard drive space eaten up by something and I have no idea what it is. I have about 80 GB free on the drive. Once I login to the computer, the amount of free space slowly starts decreasing. If I leave it alone for long enough (generally one full day), it will hit Zero KB and things will start breaking.

All I have to do is restart the computer, and suddenly I have 80 GB free again and the cycle starts all over.

Now, I have waited for it to hit Zero KB and then run both OmniDiskSweeper and WhatSize to try and determine where all my space has gone. This is the truly weird part: I can find nothing. I've looked at the drive when I've had 80 GB free and I've looked at the drive when it's been at Zero KB, and I can't see a difference. I certainly can't find anything out of the ordinary that's suddenly eating up 80 GB of my space.

I'm at my wits' end here. This has been going on for a little over a week now and I've been sure I could resolve it several times, but up to now I must admit defeat. I don't have any programs running aside from my normal startup items (background apps), and as far as I can tell, nothing has been added to my list of startup items (either in System Preferences or under /Library or ~/Library) since this problem started.

Is there anything anyone could suggest to allow me to monitor what is writing to the disk, or see where the space is going, or anything else that might help? This is a very weird situation and I've never seen anything like it before in all my years of Mac ownership. (And I'm an IT guy, too!) Any assistance would be most appreciated.

 

[Sun Baked]

Basically what apps are you running in the background all the time?

One of them might be having a memory leak or something similar.

Also checking the logs using console.app to see what they have to say might also shine some light on things.

---

If it immediately begins at login, check the login items first.

 

[timelord726]

Basically what apps are you running in the background all the time?

One of them might be having a memory leak or something similar.

That'd be quite a "memory leak" to eat 80 GB of disk space in under 24 hours.

For the sake of analysis, here are my startup items.

• GrowlHelperApp
• PastebotSync
• I Love Stars
• Air Video Server
• CoverSutra
• TextExpander
• BetterTouchTool
• Cloud
• ChaxHelperApp
• Default Folder X Helper
• Divvy
• iTunesHelper
• JustNotes
• Dropbox
• Rowmote Helper
• bzbmenu (Backblaze)

And here are my agents/daemons.

My Agents
com.apple.SafariBookmarksSyncer
ws.agile.1PasswordAgent

Users Agents
com.bjango.istatlocal

Users Daemons
com.backblaze.bzserv
com.rogueamoeba.hermes
com.vmware.launchd.vmware
noip2
org.glimmerblocker.proxy

Also checking the logs using console.app to see what they have to say might also shine some light on things.

Any recommendation as to which log / what entries would be best to check for this?

If it immediately begins at login, check the login items first.

Because it's a relatively slow process, I can't tell exactly when it begins. I have to step away for 10-15 minutes and then come back and check to see that my free space has gone down.

 

[costabunny]

open a few terminal windows and (as root) tail the system.log (in /var/log) and also keep an eye on running processes (top will show these).

You could also run a 'du -hs *' on the root dir of the disk, then about 10 mins later run it again and look for the dir increasing in size. You may have to cd to that dir and repeat this process a few times.

an ls -lart should show the files (all the file including hidden ones) and their mod times.

These tasks should help you manually find the file/s (I am a bit old school so this would be my way of hunting down the culprit).

 

[Sun Baked]

Wow just search memory leak and the login apps and you'll find stuff like

GrowlHelperApp and Air Video Server having an issue with some of the versions.

Probably a couple more of them if you look.

 

[timelord726]

Wow just search memory leak and the login apps and you'll find stuff like

GrowlHelperApp and Air Video Server having an issue with some of the versions.

Probably a couple more of them if you look.

Yeah, except I've been running these apps for months (years in the case of Growl) and never had any issues like this at all.

And I'm running these same apps (same versions) on two other computers and having no issues there.

Also, I'm still not equating the whole "memory leak" theory with hard drive space disappearing.

 

[lstone19]

Also, I'm still not equating the whole "memory leak" theory with hard drive space disappearing.

Swap space. A memory leak can cause a process's virtual memory requirement to grow and grow. To the extent that it exceeds physical memory available to it, the rest has to go on disk in swap space. That it all comes back on reboot is a good hint that its swap space.

I would run Activity Monitor and see what processes have ridiculously large virtual memory sizes.

 

[timelord726]

Swap space. A memory leak can cause a process's virtual memory requirement to grow and grow. To the extent that it exceeds physical memory available to it, the rest has to go on disk in swap space. That it all comes back on reboot is a good hint that its swap space.

I would run Activity Monitor and see what processes have ridiculously large virtual memory sizes.

I've lost 28 GB since I posted this thread this morning. Doesn't seem like the amount of virtual memory being used covers that.

 

[Sun Baked]

Turn off spotlight for right now, drag the drives to the privacy pane.

 

[timelord726]

open a few terminal windows and (as root) tail the system.log (in /var/log) and also keep an eye on running processes (top will show these).

I don't see anything immediately suspicious in the system.log.

You could also run a 'du -hs *' on the root dir of the disk, then about 10 mins later run it again and look for the dir increasing in size. You may have to cd to that dir and repeat this process a few times.

FYI, here is what that command produced when I ran it as soon as you posted it:

108K	AFM.temp18428679
 52G	Applications
4.8G	Developer
280G	Library
  0B	Network
2.6G	System
352G	Users
3.2T	Volumes
3.9M	bin
  0B	cores

And here is what it produced a few minutes ago:

108K	AFM.temp18428679
 52G	Applications
4.8G	Developer
281G	Library
  0B	Network
2.6G	System
353G	Users
3.2T	Volumes
3.9M	bin
  0B	cores

So we've had Library increase by about 1 GB and Users increase by about 1 GB. I've now lost 30 GB in this time. It doesn't seem to correlate.

(I am a bit old school so this would be my way of hunting down the culprit).

Hey, I'm totally cool with the old school approach. I'm much the same.

 

[timelord726]

Turn off spotlight for right now, drag the drives to the privacy pane.

Done. Still losing space.

 

[timelord726]

FYI, it ran down to Zero KB before I went to bed last night and had to be restarted. It was Zero KB again when I woke up this morning (though I did sleep pretty late). Restarted again and it went back to 80 GB, as usual. It's now at 27 GB free. This means it's happening faster than I initially estimated (seems to be about 12 hours for it to run down from 80 GB to Zero KB free).

 

[misee]

Although the space going back to normal after a reboot might sound like a memory leak, I find it hard to believe that a memory leak can use up 80GB of disk space. What I have experienced though, is that Flash actually does use up all of my disk space, but I was watching youtube movies in HD (although one at a time, so why?!?) - I think they (Adobes Flash team) are doing it on purpose to convince people that Apple's move not using Flash for the iPhone was the right choice
Anyway, Activity Monitor didn't report me any big memory usage either, disk space just kept disappearing as I watched more movies.

By the way, have you tried to contact Apple's support about this?

If nothing else works, I guess you can only try reformatting your hard drive and do a clean install of the system (not a time machine restore, that would probably just get you the disk space eating thingy back). Since you say you use the same applications on your other machines, it might help. Maybe it's just some strange bug in the OS that you somehow triggered on that one machine which I find hard to believe though...

Another thing that just I just thought of: have you checked the /private/tmp and /var/tmp folders? I'm not quite sure whether the contents of these folders are deleted or just moved to the trash when rebooting though, so it might be a lost thought...

 

[timelord726]

Although the space going back to normal after a reboot might sound like a memory leak, I find it hard to believe that a memory leak can use up 80GB of disk space. What I have experienced though, is that Flash actually does use up all of my disk space, but I was watching youtube movies in HD (although one at a time, so why?!?) - I think they (Adobes Flash team) are doing it on purpose to convince people that Apple's move not using Flash for the iPhone was the right choice
Anyway, Activity Monitor didn't report me any big memory usage either, disk space just kept disappearing as I watched more movies.

By the way, have you tried to contact Apple's support about this?

If nothing else works, I guess you can only try reformatting your hard drive and do a clean install of the system (not a time machine restore, that would probably just get you the disk space eating thingy back). Since you say you use the same applications on your other machines, it might help. Maybe it's just some strange bug in the OS that you somehow triggered on that one machine which I find hard to believe though...

Another thing that just I just thought of: have you checked the /private/tmp and /var/tmp folders? I'm not quite sure whether the contents of these folders are deleted or just moved to the trash when rebooting though, so it might be a lost thought...

Thanks very much for the reply.

I never did contact Apple's support about this. Honestly, I simply did not have the time to go around troubleshooting it, so doing a clean install of the OS is what I did. I haven't seen the problem repeat since then.

/private/tmp and /var/tmp were checked when I scanned the drive for used space, and I never saw them fill up, so I don't think they were the culprit. I still can't begin to tell you what was going on here. Weird, and unfortunately I don't think we'll ever know the answer.

 

[kelemi]

same situation here

Help needed!!

I am having a similar problem. the available space is decreasing automatically on its own and nothing is running..
I lost around 80GB in 20 hours in the available space info.
If I restart the computer, it will be back to normal and start decreasing again slowly on it's own.

Had monitored file size, folder size intensively , no big difference found.
But the available space just keep getting smaller and smaller, it's like an invisible file eating up free space on the hard drive.

Wondering if you ever figure out what's the reason.

 

[allan.nyholm]

You have to watch out for that "mds" process. 2,7GB of virtualmem seems overkill.

 

[Azagar]

This most definitely sounds like a log file is going haywire, could you paste a snippet of your console log.

Do you have Super Duper? This program did this to me once before, an error was repeated over and over and its console log inflated to take all my HD.

Now if it's none of the above could you go to Daisydisk.com download the program and scan your HD. Preferably around where maybe half of that 80GB have gone missing.

 

[timelord726]

Wondering if you ever figure out what's the reason.

FYI, I never did figure out the reason, and in the end I didn't have the time or patience to find out what was causing it so I ended up reinstalling Mac OS X from scratch. The problem hasn't reappeared, but I wish I could have found out what happened. I'll always be curious.