Time Machine back-ups are using up my entire external HDD

[jnyepu]

I seem to recall a post about this within the last week but when I search Time Machine and backups, the search function on macrumors tells me there is no match, so I apologize for a repeat post.

What am I doing wrong? The Leopard upgrade install went fine this AM and I left the house only to return to find sequential hourly backups of my entire internal HDD on my 500GB external. I do want it to back up the entire internal HDD, but I thought one backup would override the prior (now that I think about it, that would kind of defeat the purpose of the "time machine", wouldn't it?).

Can I configure Time Machine to do just one backup a day and/or overwrite the prior back up, so that my entire 500GB external HDD won't be overrun in less than a week? Alternatively, can I set it to automatically delete all but the newest backup, or does this have to be done manually (and can doing this manually be done safely -- seems kind of tedious to do it manually, however).

Thanks.

 

[GNice]

I posted that question...and I'm still looking for an answer...I too can't figure out why all the space on my external HD is consumed so quickly by TM. If you figure it out please post the solution.

 

[CavemanUK]

I seem to recall a post about this within the last week but when I search Time Machine and backups, the search function on macrumors tells me there is no match, so I apologize for a repeat post.

What am I doing wrong? The Leopard upgrade install went fine this AM and I left the house only to return to find sequential hourly backups of my entire internal HDD on my 500GB external. I do want it to back up the entire internal HDD, but I thought one backup would override the prior (now that I think about it, that would kind of defeat the purpose of the "time machine", wouldn't it?).

Can I configure Time Machine to do just one backup a day and/or overwrite the prior back up, so that my entire 500GB external HDD won't be overrun in less than a week? Alternatively, can I set it to automatically delete all but the newest backup, or does this have to be done manually (and can doing this manually be done safely -- seems kind of tedious to do it manually, however).

Thanks.

Well, if your Time Machine is working correctly, it should basically have done a full backup of your machine and then only backups of the files that change every hour.

If your drive is full, a couple of things could have happened. firstly and obviously, if your main HD is full or has more data than your external, then that could be the problem. The other suggestion is that large files are changing every hour. This could be something like a large download file or maybe a big database. The solution in this case is to exclude your downloads folders (and other folders with huge files) from your Time Machine backup.

Its also possible its an initial bug with Time Machine.

 

[yippy]

No, Time Machine very clearly states in its preferences that it will save and continue to make incremental backups until the backup drive is full.

If you don't like this, partition your external hard drive into a Time Machine section and an other data section. This is the ONLY way to control how much space Time Machine uses.

 

[GNice]

No, Time Machine very clearly states in its preferences that it will save and continue to make incremental backups until the backup drive is full.

If you don't like this, partition your external hard drive into a Time Machine section and an other data section. This is the ONLY way to control how much space Time Machine uses.

That's exactly what I did...but my Time Machine partition got used up VERY quickly. There is no way it's doing incremental backups (in my case). I put nothing new on my machine (just surfed the web) yet my TM machine partition filled up fast. There is something I'm missing...

 

[yippy]

How much space is used on your internal and how much is used on your Time Machine Drive?

Just so you know, if you change any files it re-backs them up. For example, if you have a DMG that you modify the contents, TM will make a new copy of the whole DMG.

Also, if you want to make it smaller you can always exclude files or folders from being backed up.

 

[GNice]

How much space is used on your internal and how much is used on your Time Machine Drive?

Just so you know, if you change any files it re-backs them up. For example, if you have a DMG that you modify the contents, TM will make a new copy of the whole DMG.

Also, if you want to make it smaller you can always exclude files or folders from being backed up.

~50GB on internal...TM partition 125GB. I'm going to figure out what to exclude and try it again...thx for the input!

 

[dapetrun]

I have limited experience here as I only installed Leopard last week but I'll attempt to explain what I think may be happening.

After the first time you connect your external drive and told Time Machine that you want to use this drive for backups, TM did a full and complete backup of your computer's hard drive to your external hard drive. This probably took awhile to accomplish (depending on how big your hard drive is). Thereafter, and for as long as you leave your computer running, (or each time you turn it on) it will do Incremental backups (meaning, only the files that have changed since the last backup) once every hour, so these Incremental backups will only take a short time to complete. TM will initiate the Full and/or Incremental backups without alerting the user and without user input. I have never left my external hard drive connected to my computer all the time so whenever I re-connect the external unit, it immediately begins to backup files that have changed since the last time the external drive was connected.

Logically, you would think it best to occasionally initiate full backups (say, once a month or every 3) and delete the older Full backup and the Incremental files associated with it. The longer the time period between the deletion of those Full/Incremental backups, the better control you have over recovery of lost or accidentally deleted files. However, the price you pay for this added control is more files and space being used on your external hard drive.

I am waiting for MacWorld's complete rundown of Time Machine to fully understand how the user can control the parameters of one's backups by some other means besides unplugging the external hard drive. I saw no way to control these parameters in my initial use but, admittedly, I spent very little time looking for it.

As others have suggested, it would be wise to NOT backup cache directories and internet history directories as these change often and become quite large very quickly. However, I cannot say HOW to tell Time Machine to NOT backup these files. Frustrating, isn't it!

Addendum: Under System Preferences under options there is a place where you can tell Time Machine what files/directories you do NOT want backed up.

 

[yudilks]

........ I have never left my external hard drive connected to my computer all the time so whenever I re-connect the external unit, it immediately begins to backup files that have changed since the last time the external drive was connected.
......

I don't quite understand here... Is this means that for example, during the first 24 hours, TM is supposed to create a backup every hour after the initial backup... But what if I don't connect my external HD all the time? What if I just reconnect my HD after the 24th hour... Will TM remember and create all the 23 incremental backups that I missed? Or perhaps it will just backup the latest incremental backup which is the 24th?

I hope everyone can understand what I mean.... I think the latter is true since it is not possible for TM to create a copy of all 23 backups inside our internal HD since it will quickly consume it...

 

[~Shard~]

For what it's worth, feel free to refer to our Time Machine FAQ here on MacRumors as there is lots of great information in there and many questions answered...

 

[dapetrun]

I don't quite understand here... Is this means that for example, during the first 24 hours, TM is supposed to create a backup every hour after the initial backup... But what if I don't connect my external HD all the time? What if I just reconnect my HD after the 24th hour... Will TM remember and create all the 23 incremental backups that I missed? Or perhaps it will just backup the latest incremental backup which is the 24th?

I hope everyone can understand what I mean.... I think the latter is true since it is not possible for TM to create a copy of all 23 backups inside our internal HD since it will quickly consume it...

Regardless how long you have the external hard drive disconnected, when you next connect the drive it will do (one) Incremental backup of only the files that have changed since the last backup. But, if you leave the computer on for 5 hours and go shopping during that time, it will likely complete 5 Incremental backups. However, since you didn't do anything in the last 5 hours, there will be next to nothing backed up because nothing has changed in those 5 hours.

 

[jnyepu]

Regardless how long you have the external hard drive disconnected, when you next connect the drive it will do (one) Incremental backup of only the files that have changed since the last backup. But, if you leave the computer on for 5 hours and go shopping during that time, it will likely complete 5 Incremental backups. However, since you didn't do anything in the last 5 hours, there will be next to nothing backed up because nothing has changed in those 5 hours.

So this is what I don't understand. I left the computer on for 4 hours (NOT in sleep mode), left the house, and when I came back there were 4 sequential backups of my internal HDD on the hour taking up room on my external HDD. (BTW, my internal has only 60.8GB used and close to 90GB free). Of these 4 back ups, the first (earliest) was only 669MB, the second was 1.56GB, the third increased to an amazing 57.43GB and the fourth was also 57.43GB. The computer was idle that whole time so I don't know what changed.

I don't understand why the first two were such small backups and the last two were closer to the amount of data on my internal HDD?

Incidentally, before I left the house again for about 4 hours I put the computer in sleep mode and NO backups were made during that period.

Can I safely delete the 1st three backups I described above and just keep the 4th "Latest" in Time Machine terms? Then, I'll go into preferences and be more selective with what Time Machine backs up, and see what happens.

ADDENDUM:I just deleted the three earlier backups and everything seems OK so far.

 

[lilredmacbook]

I've actually disabled time machine on both of my macs, because the headaches involved at this point have just made it virtually useless.

first - on my macbook - i'm using more than 50% of my internal drive, and rarely have it in one place enough to hook it up to an external, so as cool as this feature seems, it doesn't seem to have much practicality for those of us who keep their laptops ultra-mobile. it'd be great if you could just save the incremental backups internally and then "download" them to an external that already has the base save.

second - on my macpro - if i were to back up everything on my internal drives, and then expect to do incremental backups, then i'd be looking at at least 1.8 tb of space just to start with. also, i'm primarily working with HD footage in FCP. when I turned time machine on, the processing speed of the entire machine slowed to a point that HD playback was virtually impossible. it was like i was on a G4! and this happened every hour, when the incrementals would kick in. at that point, i just had it backing up my Mac HD, and it still seemed to take forever.

for the moment i'm just sticking to weekly backups with synk. TM just strikes me as a little too buggy for my full faith quite yet.

 

[GNice]

~50GB on internal...TM partition 125GB. I'm going to figure out what to exclude and try it again...thx for the input!

I hope I figured what may be at a issue in my case. I'm running VMWare...the VM file is about 15GB. Unfortunately, I have to use XP (for work) a bit each day. Not sure if it was the culprit, but if that file was being backed up with the incrementals than it wouldn't take long to use up a significant amount of space. I've excluded it...we'll see if that helps.

 

[Macsterling]

Wirelessly posted (iPhone: Mozilla/5.0 (iPhone; U; CPU like Mac OS X; en) AppleWebKit/420.1 (KHTML, like Gecko) Version/3.0 Mobile/3A109a Safari/419.3)

I bought a 500 gb external for time machine and other things. After I did the first intial backup then I just turned it off in the prefferances and now will only back up when I want it to by turning it on

 

[CavemanUK]

I hope I figured what may be at a issue in my case. I'm running VMWare...the VM file is about 15GB. Unfortunately, I have to use XP (for work) a bit each day. Not sure if it was the culprit, but if that file was being backed up with the incrementals than it wouldn't take long to use up a significant amount of space. I've excluded it...we'll see if that helps.

yes, this could very easily cause the problem. This instant that file changes, you would have another 15Gb backup.

Just as a FYI for everyone reading the thread that arent completely familiar with what Time Machine does. (Im doing this from memory so i may not be 100% accurate)..

If you were to examine your external drive, you would see a folder called something like backup.backupdb

Inside here you will see folders relating to the time and date if your hourly backups.. like so..

20070201-1510 55gb
20070201-1610 56gb

Now this doesnt mean that the total space used on your drive is 111Gb. The file system is clever enough so that if the files are identical they ocupy the same physical space on the external drive. So if only 1Gb worth of files have changed, the real size of the second folder is really on 1Gb but the file system treats it as a real folder with a complete backup.

hope that made sense... its a very cool concept and mechanism.

The only drawback as discussed above is that if a file changes, it has to re-save that complete file. When ZFS comes in, this should be able Time Machine to only have to backup the actual parts of a file that have changed.

 

[bankshot]

I hope I figured what may be at a issue in my case. I'm running VMWare...the VM file is about 15GB. Unfortunately, I have to use XP (for work) a bit each day. Not sure if it was the culprit, but if that file was being backed up with the incrementals than it wouldn't take long to use up a significant amount of space. I've excluded it...we'll see if that helps.

Yep, that'd definitely do it. This is one reason why many of us were lamenting the fact that Apple didn't have enough time to fully integrate ZFS with read-write support before release. Among many other things, that could have enabled Time Machine to do incremental backups on the block level, copying only the changed blocks within such massive files (just as CavemanUK said). There are quite a few other exciting ways that ZFS could really improve Time Machine, but I doubt we'll see most (if any) of them before 10.6. I'm sure read-write ZFS will come along in a 10.5.x update, but who knows if Time Machine will take advantage then?

Regardless, it would be nice if Apple could add an option to Time Machine to only keep the most recent version of certain files in the backups. This would be especially useful for Parallels disk images, which do their own internal snapshotting (does VMWare do this too?). As long as the file itself doesn't get corrupted, you already have a good history to go "back in time." Until/unless Apple adds this, I guess that's where other backup programs pick up the slack (or just remember to do a manual copy every one in a while!).

I haven't gotten Leopard yet (waiting for .1 or .2) but I know I'm going to have to figure out an alternate strategy for my Parallels images.

 

[psonice]

If you have vmware drives saved on your main disk, they'll get backed up all the time and it'll fill your backup in no time. Exclude the vmware files so they don't get backed up again, then open time machine and find the big files. Select them, then click on the tools button in the finder (the one with the cog on). You can select 'delete all backups of this file' or something to that effect, and recover all the space.

Also, don't believe finder when you ask it how big the backup folders are. All later folders link to the original, so they can all report that they take 50gb when in fact they might only take a couple of mb.

 

[jnyepu]

yes, this could very easily cause the problem. This instant that file changes, you would have another 15Gb backup.

Just as a FYI for everyone reading the thread that arent completely familiar with what Time Machine does. (Im doing this from memory so i may not be 100% accurate)..

If you were to examine your external drive, you would see a folder called something like backup.backupdb

Inside here you will see folders relating to the time and date if your hourly backups.. like so..

20070201-1510 55gb
20070201-1610 56gb

Now this doesnt mean that the total space used on your drive is 111Gb. The file system is clever enough so that if the files are identical they ocupy the same physical space on the external drive. So if only 1Gb worth of files have changed, the real size of the second folder is really on 1Gb but the file system treats it as a real folder with a complete backup.

hope that made sense... its a very cool concept and mechanism.

The only drawback as discussed above is that if a file changes, it has to re-save that complete file. When ZFS comes in, this should be able Time Machine to only have to backup the actual parts of a file that have changed.

Thanks for the clarification. This makes more sense to me now. I do run VMware as well, so I will also Exclude it from the backup and see what happens. In retrospect, deleting the earlier backups may have been a bad idea if/when it comes time to restore, but the oldest backup of two I have in the backup.backupdb file seems to be the 57 GB range, which is about right.

The other option as posted above is to just disable TM and only enable it when I want a backup. This might be a good way to go until Apple works out the kinks in TM and maybe makes it more user-customizable.

 

[jnyepu]

I hope I figured what may be at a issue in my case. I'm running VMWare...the VM file is about 15GB. Unfortunately, I have to use XP (for work) a bit each day. Not sure if it was the culprit, but if that file was being backed up with the incrementals than it wouldn't take long to use up a significant amount of space. I've excluded it...we'll see if that helps.

BTW, what files did you Exclude for VMWare Fusion? Just want to make sure I've hit all the same ones.

Thanks.

 

[juanster]

i had some problems like this, i have a 500 gb external (partitioned in half for TM)and my internal is only 80, the first week or so TM used it all up i only had like 3GB left i was freaking out, but since i did not know what to do i let it be. the next day it somehow went to 131 gb free. and all my back up was still there. now it has been maybe a week or so since then, maybe more and it's at 129 free, seems like a good deal to me i have added lots of pictures, edited lots of pictures and added many songs..

 

[Pierce]

I just wish it had an option when to do a scheduled backup, ie at 3:00am once a week when me or my family is not using the computer.

I also dont like it when TM is on and I have Photoshop, Illustrator and several other heavy applications open. It seems to affect the performance a tad.

For now I just do a backup, Turn off TM, wait for about a week and then turn on TM and back it up . Kind of silly it do it this way but there is no other option.

 

[RafMac]

Time Machine Tip

I noticed that if you turn another external HD, Time machine will back this up too. You can ad this HD in Time Machine preferences not to back up. I also suggest not to back up 'applications' folder as this is big. If one app or more fails, all you do is reinstall that app only. Also system folder doesnt' need to backed up either. Back up only what you need.

 

[elistan]

So this is what I don't understand. I left the computer on for 4 hours (NOT in sleep mode), left the house, and when I came back there were 4 sequential backups of my internal HDD on the hour taking up room on my external HDD. (BTW, my internal has only 60.8GB used and close to 90GB free). Of these 4 back ups, the first (earliest) was only 669MB, the second was 1.56GB, the third increased to an amazing 57.43GB and the fourth was also 57.43GB. The computer was idle that whole time so I don't know what changed.

Keep in mind that if only two files on the entire HD change between the 3pm and 4pm, when TM makes a backup of those files, it will look like the ENTIRE HD is backed up. But it wasn't, since TM is smart enough to make the restore image link to previous backups of those files that did not change.

Of course, if those two files are 15GB virtual machines like stated below, yeah, you'll fill a backup drive quickly. 🙂

 

[GNice]

BTW, what files did you Exclude for VMWare Fusion? Just want to make sure I've hit all the same ones.

Thanks.

<VMName>.vmwarevm file located in /Users/<username>/documents/Virtual Machines