Time Machine backup smaller than it should

[NunoBerny]

I've been having problems doing backups with my new WD Elements 1TB USB3.
The drive is probably at fault (failed backups for no apparent reason, spontaneous disconnection) and tomorrow I'll try to swap it, but there's one other thing that is bugging me:
-the TM prefs says the full backup should be 300GB+, but the ongoing operation (one hour to go, usually fails before that...) is doing a 193GB backup!!!
Where are the "other" 107GB?!??!
Even if the target drive is faulty, the TM program seems to be misbehaving.

Anyone had something like this happen?
It's driving me crazy for a couple of weeks now.

Mac Mini, Mountain Lion 10.8.3, 8GB RAM

 

[NunoBerny]

I'm seriously thinking about quitting on Time Machine.

If they agree to give me a new drive (I'll have to prove it's faulty, which is no sure thing), I'll give that one a try, but after that I don't know.

 

[Weaselboy]

I'm seriously thinking about quitting on Time Machine.

If they agree to give me a new drive (I'll have to prove it's faulty, which is no sure thing), I'll give that one a try, but after that I don't know.

Try two things after you get the new drive. Do a command-r boot to recovery then run Disk Util and do a repair disk on your main drive. Then restart and run the command below in Terminal to rebuild the Spotlight index. Sometimes the Spotlight index is wanky and TM relies on that to ID files.

sudo mdutil -E /

The total TM backup will be somewhat less than your total drive data because TM does not backup some cache files, but it should not be different by as much as you are seeing.

 

[NunoBerny]

Try two things after you get the new drive. Do a command-r boot to recovery then run Disk Util and do a repair disk on your main drive. Then restart and run the command below in Terminal to rebuild the Spotlight index. Sometimes the Spotlight index is wanky and TM relies on that to ID files.

sudo mdutil -E /

The total TM backup will be somewhat less than your total drive data because TM does not backup some cache files, but it should not be different by as much as you are seeing.

In this user, I can't use "sudo" (for some reason). I changed to another and it got it going. Not sure if indexing one user affects the others.
I did a few verify/repair on the main disk, but nothing is reported (aside form the occasional "permission" repair).

I expected the TM backup to be slightly smaller, but not by almost 50%.

 

[Weaselboy]

In this user, I can't use "sudo" (for some reason). I changed to another and it got it going. Not sure if indexing one user affects the others.
I did a few verify/repair on the main disk, but nothing is reported (aside form the occasional "permission" repair).

I expected the TM backup to be slightly smaller, but not by almost 50%.

No matter which acct. you do the reindex from, it will do the whole drive. Hopefully this helps.

 

[NunoBerny]

No matter which acct. you do the reindex from, it will do the whole drive. Hopefully this helps.

I'll report as soon as I have the new drive ready.
(crossing fingers)

 

[NunoBerny]

UPDATE:

Something is wrong with my system.
Time Machine is NOT working as it should.
I swapped my WD Elements USB drive, but TM still cannot complete the backup.
And it still reports it's making a 250GB, while the TM prefs stated the full backup would be 380GB.

Is there a way to reset everything TM related (as if I'm making a backup on a new computer)? I already deleted the prefs file, but nothing improved.
It seems some fault is being kept from the previous problems I had (with the other drive). I'm hoping is not something in my main HD, since it never displayed any problem. The only problem is indeed when making a backup with TM.

I'm a bit lost here...

 

[Weaselboy]

UPDATE:

Something is wrong with my system.
Time Machine is NOT working as it should.
I swapped my WD Elements USB drive, but TM still cannot complete the backup.
And it still reports it's making a 250GB, while the TM prefs stated the full backup would be 380GB.

Is there a way to reset everything TM related (as if I'm making a backup on a new computer)? I already deleted the prefs file, but nothing improved.
It seems some fault is being kept from the previous problems I had (with the other drive). I'm hoping is not something in my main HD, since it never displayed any problem. The only problem is indeed when making a backup with TM.

I'm a bit lost here...

If you delete the prefs file and have erased the external drive to start over, that should work.

Try the steps in this older post to see if that helps. This will reset your user permissions and ACLs and has helped others with TM troubles.

 

[NunoBerny]

If you delete the prefs file and have erased the external drive to start over, that should work.

Try the steps in this older post to see if that helps. This will reset your user permissions and ACLs and has helped others with TM troubles.

I've deleted the prefs and erased and repartitioned the drive, for good measure. Still the backup failed.
And what's really annoying is that it usually fails more than halfway through doing the backup, which means I'm waiting one hour or more for nothing, making the trial-and-error extremely frustrating and time consuming.

I've also fixed the permissions on the main HD.
I've never heard of the ACL, will try that after the current backup ends/fails.

New info, probably relevant:
I've excluded a few folders from the backup, see if it helps. The first try, with only an external (100GB) USB drive went well. Now I'm trying with all my main HD folders EXCEPT the system files.
If this one (280GB) succeeds, the problem should be on my various system files, the only one left out of this backup.
This will make it harder for me, since I have no idea how to install from scratch, if needed.
I also have the new Mountain Lion update waiting, never did it because...I don't have a backup!

I still find it strange that a supposed 480GB "becomes" about 200GB when TM starts the backup... ;(

Anyway, thanks for your help and sorry about the long post. I'll report back as soon as possible.

 

[NunoBerny]

UPDATE:

Booted through Recovery Disk:

The disk utility tool did indeed reveal that there was one ACL problem in Applications. I repaired it and then also did the "reset home folder permissions and ACLs section", through Terminal.

Created a fresh partition for Time Machine, but only later will I try to backup again, will let it running while I'm away. Fingers crossed.

One more question: on the recovery disk options, if I do a base OSX install, will I lose everything on the system HD or it only messes with the OS files?
I'm referring to losing prefs, documents or applications that I have stored there (without a backup, for the reasons you already know...).


PS: so far, I'm a bit disappointed with Time Machine. It seems to go against the "it just works" Apple mantra. It has made my life very difficult, I'll tell you. ;(

 

[Weaselboy]

One more question: on the recovery disk options, if I do a base OSX install, will I lose everything on the system HD or it only messes with the OS files?
I'm referring to losing prefs, documents or applications that I have stored there (without a backup, for the reasons you already know...).(

If you just reinstall the OS without first using Disk Util to erase the drive, it will not overwrite your data/documents etc.

That said, I hate to see you do this without a backup. There is always the possibility of the install process stressing a borderline drive and causing it to die.

 

[NunoBerny]

If you just reinstall the OS without first using Disk Util to erase the drive, it will not overwrite your data/documents etc.

That said, I hate to see you do this without a backup. There is always the possibility of the install process stressing a borderline drive and causing it to die.

I hate it too... Even though I don't have any vital data, I'd be mad if I lost everything.
I can always put everything on the USB drive, for safety reasons. If nothing solves my problem and I have to re-install, I'll copy everything there.
Still, I'm confident that this time the backup will work!

My lack of knowledge about OSX will hurt me, obviously. I have little knowledge about which files go where (for example, saved games and application-related data), but am improving on that front.

 

[NunoBerny]

Update:

It was not to be. The backup still failed. When I got home, the dreaded message was on the screen. ;(
Still, it's now CONTINUING the backup. I'll wait and see if it ends this time.

I don't know if the machine going to sleep midway could be at fault, even though I have it set to "not go to sleep".


Update 2:
The "second" backup ended, but Time Machine seems happy to do only a 190GB backup, when the TM prefs said the full backup would supposedly be 385GB.
Why the hell does it do such a smaller backup?!? What does it do (or not do) with the remaining 200GB?
And why no complain, no checking if the values match?

I'm really frustrated.
Whatever the problem, Time Machine is very unhelpful. I can't trust a program like this. I'm probably going to quit on it... ;(

 

[NunoBerny]

Most of this is chinese to me, but maybe it's relevant to finding out what's happening.

Console report:

7/13/13 9:11:26.741 PM System Preferences[1646]: Could not connect the action backUpOnBatteryPowerPressed: to target of class TMOptionsSheetController
7/13/13 9:11:37.933 PM com.apple.backupd[1657]: Starting manual backup
7/13/13 9:11:38.497 PM com.apple.backupd[1657]: Backing up to: /Volumes/TimeMachine/Backups.backupdb
7/13/13 9:11:38.499 PM com.apple.backupd[1657]: Ownership is disabled on the backup destination volume. Enabling.
7/13/13 9:11:43.077 PM com.apple.backupd[1657]: Forcing deep traversal on source: "Macintosh HD" (mount: '/' fsUUID: B98CDBD4-3E24-3B7A-8EBA-F7BE71FE76B6 eventDBUUID: BB63BC41-43FB-46C2-B439-8130BDC2E50A)
7/13/13 9:15:34.523 PM com.apple.backupd[1657]: Backup content size: 386.6 GB excluded items size: 198.57 GB for volume Macintosh HD
7/13/13 9:15:34.543 PM com.apple.backupd[1657]: Found 1357262 files (188.03 GB) needing backup
7/13/13 9:15:34.545 PM com.apple.backupd[1657]: 225.64 GB required (including padding), 999.19 GB available
7/13/13 9:15:35.017 PM com.apple.backupd[1657]: Waiting for index to be ready (100)
7/13/13 10:11:44.423 PM com.apple.backupd[1657]: Copied 85.71 GB of 188.03 GB, 901245 of 1357262 items
7/13/13 10:20:14.821 PM com.apple.backupd[1657]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Civilization IV/cache/CIV4DiplomacyInfos.dat to (null)
7/13/13 10:23:01.440 PM com.apple.backupd[1657]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/Assassin's Creed 2/Assassin's Creed 2.app/Contents/Resources/DataMAC_Mountains.forge to (null)
7/13/13 10:23:50.432 PM com.apple.backupd[1657]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/Assassin's Creed 2/Assassin's Creed 2.app/Contents/Resources/media/Movies/bink_uplaysplash.bik to (null)
7/13/13 10:24:44.902 PM com.apple.backupd[1657]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/Assassin's Creed 2/Assassin's Creed 2.app/Contents/Resources/sounddata/Mac/sounds_ita.pck to (null)
7/13/13 10:25:28.726 PM com.apple.backupd[1657]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/Assassin's Creed 2/Assassin's Creed 2.app/Contents/Resources/sounddata/Mac/sounds_rus.pck to (null)
7/13/13 10:26:29.108 PM com.apple.backupd[1657]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/Assassin's Creed 2/Assassin's Creed 2.app/Contents/Resources/sounddata/Mac/sounds_sfx.pck to (null)
7/13/13 10:27:16.455 PM com.apple.backupd[1657]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/Assassin's Creed 2/Assassin's Creed 2.app/Contents/Resources/videos/ADB_OrsiBrothers.bik to (null)
7/13/13 10:27:52.012 PM com.apple.backupd[1657]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/Assassin's Creed 2/Assassin's Creed 2.app/Contents/Resources/videos/ADB_Pazzi.bik to (null)
7/13/13 10:28:49.549 PM com.apple.backupd[1657]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/Assassin's Creed 2/Assassin's Creed 2.app/Contents/Resources/videos/ADB_Silvio.bik to (null)
7/13/13 10:30:07.510 PM com.apple.backupd[1657]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/Assassin's Creed 2/Assassin's Creed 2.app/Contents/Resources/videos/DemoIntro.bik to (null)
7/13/13 10:37:28.212 PM com.apple.backupd[1657]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/left 4 dead/left4dead/media/l4d_intro.bik to (null)
7/13/13 10:38:09.825 PM com.apple.backupd[1657]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/left 4 dead/left4dead/media/l4d_intro.mov to (null)
7/13/13 10:43:55.334 PM com.apple.backupd[1657]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/left 4 dead 2/left4dead2/maps/c4m2_sugarmill_a.nav to (null)
7/13/13 10:45:05.482 PM com.apple.backupd[1657]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/left 4 dead 2/left4dead2/media/l4d2_intro.mov to (null)
7/13/13 10:49:01.145 PM com.apple.backupd[1657]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/left 4 dead 2/left4dead2/sound/ambient/creatures/town_scared_breathing2.wav to (null)
7/13/13 10:50:00.143 PM com.apple.backupd[1657]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/left 4 dead 2/left4dead2/sound/music/zombat/danger/saw/saw_danger_02_12.wav to (null)
7/13/13 10:51:32.993 PM com.apple.backupd[1657]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/left 4 dead 2/left4dead2/sound/player/survivor/voice/gambler/battlecrytank06.wav to (null)
7/13/13 10:53:44.945 PM com.apple.backupd[1657]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/left 4 dead 2/left4dead2/sound/player/survivor/voice/producer/worldc3m1b09.wav to (null)
7/13/13 10:54:30.369 PM com.apple.backupd[1657]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/left 4 dead 2/left4dead2_dlc1/scenes/biker/dlc1_c6m1_initialmeeting15.vcd to (null)
7/13/13 10:55:11.050 PM com.apple.backupd[1657]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/left 4 dead 2/left4dead2_dlc1/scenes/biker/templates/template_surprise.vcd to (null)
7/13/13 11:01:04.625 PM com.apple.backupd[1657]: Stopping backup.
7/13/13 11:01:04.625 PM com.apple.backupd[1657]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/left 4 dead 2/left4dead2_dlc3/pak01_022.vpk to (null)
7/13/13 11:01:05.013 PM com.apple.backupd[1657]: Copied 1144019 files (121 GB) from volume Macintosh HD.
7/13/13 11:01:05.013 PM com.apple.backupd[1657]: Copy stage failed with error:11
7/13/13 11:01:15.574 PM com.apple.backupd[1657]: Backup failed with error: 11
7/13/13 11:11:52.356 PM com.apple.backupd[1657]: Starting automatic backup
7/13/13 11:11:52.367 PM com.apple.backupd[1657]: Backing up to: /Volumes/TimeMachine/Backups.backupdb
7/13/13 11:11:53.889 PM com.apple.backupd[1657]: Forcing deep traversal on source: "Macintosh HD" (mount: '/' fsUUID: B98CDBD4-3E24-3B7A-8EBA-F7BE71FE76B6 eventDBUUID: BB63BC41-43FB-46C2-B439-8130BDC2E50A)
7/13/13 11:11:54.115 PM com.apple.backupd[1657]: Found 1357262 files (67.03 GB) needing backup
7/13/13 11:11:54.117 PM com.apple.backupd[1657]: 81.13 GB required (including padding), 877.29 GB available
7/13/13 11:12:34.153 PM com.apple.backupd[1657]: Stopping backup.
7/13/13 11:12:34.153 PM com.apple.backupd[1657]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/left 4 dead 2/left4dead2_dlc3/pak01_022.vpk to (null)
7/13/13 11:12:34.475 PM com.apple.backupd[1657]: Copied 458 files (14.7 MB) from volume Macintosh HD.
7/13/13 11:12:34.475 PM com.apple.backupd[1657]: Copy stage failed with error:11
7/13/13 11:12:44.805 PM com.apple.backupd[1657]: Backup failed with error: 11
7/14/13 12:11:53.033 AM com.apple.backupd[3486]: Starting automatic backup
7/14/13 12:11:54.996 AM com.apple.backupd[3486]: Backing up to: /Volumes/TimeMachine/Backups.backupdb
7/14/13 12:11:58.986 AM com.apple.backupd[3486]: Forcing deep traversal on source: "Macintosh HD" (mount: '/' fsUUID: B98CDBD4-3E24-3B7A-8EBA-F7BE71FE76B6 eventDBUUID: BB63BC41-43FB-46C2-B439-8130BDC2E50A)
7/14/13 12:11:59.062 AM com.apple.backupd[3486]: Found 1357262 files (67.03 GB) needing backup
7/14/13 12:11:59.065 AM com.apple.backupd[3486]: 81.13 GB required (including padding), 877.29 GB available
7/14/13 12:12:36.699 AM com.apple.backupd[3486]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/left 4 dead 2/left4dead2_dlc3/pak01_022.vpk to (null)
7/14/13 12:12:53.540 AM System Preferences[3498]: Could not connect the action backUpOnBatteryPowerPressed: to target of class TMOptionsSheetController
7/14/13 12:26:50.370 AM com.apple.backupd[3486]: Error: (-36) SrcErr:YES Copying /Users/bernie/Library/Application Support/Steam/SteamApps/common/Sid Meier's Civilization IV Warlords/Warlords/Assets/Art/Leaderheads/Brennus/brennus.nif to (null)
7/14/13 12:47:16.634 AM com.apple.backupd[3486]: Error: (-36) SrcErr:YES Copying /Users/bernie/Music/iTunes/iTunes Media/Music/Elton John/The Very Best Of Elton John [Disc 2]/2-12 Nikita.m4a to (null)
7/14/13 12:57:39.350 AM com.apple.backupd[3486]: Error: (-36) SrcErr:YES Copying /Users/bernie/Pictures/iPhoto Library.photolibrary/Masters/2013/03/01/20130301-220548/1300.jpg to (null)
7/14/13 12:58:18.710 AM com.apple.backupd[3486]: Error: (-36) SrcErr:YES Copying /Users/bernie/Pictures/iPhoto Library.photolibrary/Masters/2013/03/01/20130301-220548/1318.jpg to (null)
7/14/13 12:59:00.557 AM com.apple.backupd[3486]: Error: (-36) SrcErr:YES Copying /Users/bernie/Pictures/iPhoto Library.photolibrary/Masters/2013/03/01/20130301-220548/1393.jpg to (null)
7/14/13 1:05:54.457 AM com.apple.backupd[3486]: Copied 186910 files (67.41 GB) from volume Macintosh HD.
7/14/13 1:05:54.625 AM com.apple.backupd[3486]: Using file event preflight for Macintosh HD
7/14/13 1:05:56.019 AM com.apple.backupd[3486]: Will copy (6.8 MB) from Macintosh HD
7/14/13 1:05:56.026 AM com.apple.backupd[3486]: Found 188 files (7.9 MB) needing backup
7/14/13 1:05:56.042 AM com.apple.backupd[3486]: 1.04 GB required (including padding), 809.03 GB available
7/14/13 1:06:12.356 AM com.apple.backupd[3486]: Copied 2205 files (7.1 MB) from volume Macintosh HD.
7/14/13 1:06:12.705 AM com.apple.backupd[3486]: Created new backup: 2013-07-14-010612
7/14/13 1:06:13.587 AM com.apple.backupd[3486]: Copying OS X Recovery set
7/14/13 1:06:31.996 AM com.apple.backupd[3486]: Starting post-backup thinning
7/14/13 1:06:32.037 AM com.apple.backupd[3486]: Error: Error Domain=NSOSStatusErrorDomain Code=-50 "The operation couldn’t be completed. (OSStatus error -50.)" (paramErr: error in user parameter list) deleting backup: /Volumes/TimeMachine/Backups.backupdb/Nuno’s Mac mini/2013-07-13-211143.inProgress/DF29C1A0-91BE-4A3A-A13B-C7904885A297
7/14/13 1:06:32.037 AM com.apple.backupd[3486]: No post-back up thinning needed: no expired backups exist
7/14/13 1:06:32.101 AM com.apple.backupd[3486]: Backup completed successfully.
7/14/13 1:11:22.116 AM com.apple.backupd-helper[4066]: Not starting scheduled Time Machine backup - less than 10 minutes since last backup completed.
7/14/13 1:14:35.607 AM com.apple.backupd[4091]: Starting manual backup
7/14/13 1:14:36.015 AM com.apple.backupd[4091]: Backing up to: /Volumes/TimeMachine/Backups.backupdb
7/14/13 1:14:38.567 AM com.apple.backupd[4091]: Using file event preflight for Macintosh HD
7/14/13 1:14:38.611 AM com.apple.backupd[4091]: Will copy (2.7 MB) from Macintosh HD
7/14/13 1:14:39.123 AM com.apple.backupd[4091]: Found 47 files (2.7 MB) needing backup
7/14/13 1:14:39.125 AM com.apple.backupd[4091]: 1.04 GB required (including padding), 808.57 GB available
7/14/13 1:14:46.358 AM com.apple.backupd[4091]: Copied 980 files (2.7 MB) from volume Macintosh HD.
7/14/13 1:14:46.380 AM com.apple.backupd[4091]: Using file event preflight for Macintosh HD
7/14/13 1:14:46.384 AM com.apple.backupd[4091]: Will copy (34 KB) from Macintosh HD
7/14/13 1:14:46.385 AM com.apple.backupd[4091]: Found 11 files (34 KB) needing backup
7/14/13 1:14:46.386 AM com.apple.backupd[4091]: 985.1 MB required (including padding), 808.56 GB available
7/14/13 1:14:47.718 AM com.apple.backupd[4091]: Copied 543 files (34 KB) from volume Macintosh HD.
7/14/13 1:14:47.809 AM com.apple.backupd[4091]: Created new backup: 2013-07-14-011447
7/14/13 1:14:48.183 AM com.apple.backupd[4091]: Starting post-backup thinning
7/14/13 1:14:48.183 AM com.apple.backupd[4091]: No post-back up thinning needed: no expired backups exist
7/14/13 1:14:48.220 AM com.apple.backupd[4091]: Backup completed successfully.
7/14/13 1:16:42.509 AM System Preferences[4107]: Could not connect the action backUpOnBatteryPowerPressed: to target of class TMOptionsSheetController

 

[Weaselboy]

Most of this is chinese to me, but maybe it's relevant to finding out what's happening.

Console report:

Those "error 36" entries mean TM is not copying that data form some reason. It could be the data is corrupt, or it could be random data copy errors caused by a bad drive or even bad cables.

For example, it looks like there are a bunch of errors trying to backup data in the /Users/bernie/Library/Application Support/Steam/SteamApps/ folders. Add that folder to the exclude backup section in TM settings. Do the same for any folder you see in the logs with that error. Then do another backup. Then look in the logs to see if the errors are gone. If they are gone, that tells you the data in that folder may be corrupt. If those errors for that folder are gone, but new folders show up with the error, that tells us you have a more general data corruption error.

I don't think you have anything wrong with Time Machine itself. I think you have either corrupted data or a bad drive/cable possibly.

Give this a look.

 

[NunoBerny]

Those "error 36" entries mean TM is not copying that data form some reason. It could be the data is corrupt, or it could be random data copy errors caused by a bad drive or even bad cables.

For example, it looks like there are a bunch of errors trying to backup data in the /Users/bernie/Library/Application Support/Steam/SteamApps/ folders. Add that folder to the exclude backup section in TM settings. Do the same for any folder you see in the logs with that error. Then do another backup. Then look in the logs to see if the errors are gone. If they are gone, that tells you the data in that folder may be corrupt. If those errors for that folder are gone, but new folders show up with the error, that tells us you have a more general data corruption error.

I don't think you have anything wrong with Time Machine itself. I think you have either corrupted data or a bad drive/cable possibly.

Give this a look.

I already read it many times, great info there, but the problems reported are always a little bit different, couldn't find an exact match. Tried many of the suggestions, but I'm still stuck.

I noticed the Steam apps errors. I'll exclude them and see if it helps. The way Steam works, always updating and downloading stuff, I'd say errors are bound to happen.

Important: I searched in Finder, but there's no such folder in Users/Bernie!!!! Could it be hidden?

Still, there is something wrong, since TM thinks only a fraction of the main HD needs backing up (there's a line saying only X GB need backup), so I'm assuming the remaining are "marked" as already backed up.
I wonder if there's a list stored somewhere that is (wrongly) informing TM of that.
I want to make a full backup, from scratch, yet TM always thinks it's doing a partial one.

I'll be away for 2 weeks, on vacation. Since you're the only one that offered to help so far, I'd like to ask you are willing to continue guiding me after I return.
I don't think I'll be able to do anything before I leave. ;(

 

[Weaselboy]

I already read it many times, great info there, but the problems reported are always a little bit different, couldn't find an exact match. Tried many of the suggestions, but I'm still stuck.

I noticed the Steam apps errors. I'll exclude them and see if it helps. The way Steam works, always updating and downloading stuff, I'd say errors are bound to happen.

Important: I searched in Finder, but there's no such folder in Users/Bernie!!!! Could it be hidden?

Still, there is something wrong, since TM thinks only a fraction of the main HD needs backing up (there's a line saying only X GB need backup), so I'm assuming the remaining are "marked" as already backed up.
I wonder if there's a list stored somewhere that is (wrongly) informing TM of that.
I want to make a full backup, from scratch, yet TM always thinks it's doing a partial one.

I'll be away for 2 weeks, on vacation. Since you're the only one that offered to help so far, I'd like to ask you are willing to continue guiding me after I return.
I don't think I'll be able to do anything before I leave. ;(

I would be glad to continue helping you out. Just bump this thread when you get back or PM me and we can pick up where we left off in the thread. Enjoy the vacation.

/Users/bernie is your personal home folder. When you open Finder you should by default be in that folder.

Or... open Finder then hit shift-command-g (all three at once) and paste in /Users/bernie then hit enter and you will see all your user files. The user Library folder is hidden by default, but you can see it by option clicking the Go menu while in Finder.

7/13/13 9:15:34.523 PM com.apple.backupd[1657]: Backup content size: 386.6 GB excluded items size: 198.57 GB for volume Macintosh HD

This line in that log tells me something is being excluded from the backup. There are three ways this can happen.

1. OS X automatically excludes some cache files that don't need to be backed up.

2. Folders you manually add the TM exclude pref pane.

3. Apps can add exclusions to the Time Machine plist file.

To see what #3 is up to you can use Quicklook to peek in the file at /Library/Preferences/com.apple.TimeMachine.plist and look for entries under <key>Excludebypath<key>. Say a developer makes an app that creates a bunch of temp files that don't need to be backed up, the dev. can have his app add this entry to the TM plist to prevent needless backup of the temp files.

 

[NunoBerny]

I would be glad to continue helping you out. Just bump this thread when you get back or PM me and we can pick up where we left off in the thread. Enjoy the vacation.

/Users/bernie is your personal home folder. When you open Finder you should by default be in that folder.

Or... open Finder then hit shift-command-g (all three at once) and paste in /Users/bernie then hit enter and you will see all your user files. The user Library folder is hidden by default, but you can see it by option clicking the Go menu while in Finder.

This shows you the extent of my current OSX knowledge: I didn't know how to do that.

This line in that log tells me something is being excluded from the backup. There are three ways this can happen.

1. OS X automatically excludes some cache files that don't need to be backed up.

2. Folders you manually add the TM exclude pref pane.

3. Apps can add exclusions to the Time Machine plist file.

To see what #3 is up to you can use Quicklook to peek in the file at /Library/Preferences/com.apple.TimeMachine.plist and look for entries under <key>Excludebypath<key>. Say a developer makes an app that creates a bunch of temp files that don't need to be backed up, the dev. can have his app add this entry to the TM plist to prevent needless backup of the temp files.

It would be ok if OSX excluded a couple of files, as it used in the beginning, but not that many GB.
As for folders manually excluded by, I only added the 2 external USB HDs, nothing from the main HD.
The #3 could be it, but I did delete the TimeMachine.plist yesterday and the GB continued to be excluded. This is what makes me think if there is another "mysterious" file housing these TM instructions.

 

[NunoBerny]

I also thought about installing the new Mountain Lion update and hope it would magically cure the problem, but without a backup that's probably a risk.

By the way, if I have corrupted files, how can I detect them? I'd expect the Disk Utility tool took care of this, but it seems it doesn't, at least not completely.

 

[Weaselboy]

The #3 could be it, but I did delete the TimeMachine.plist yesterday and the GB continued to be excluded. This is what makes me think if there is another "mysterious" file housing these TM instructions.

Yes, but if it is the #3 option, and say Steam tells TM not to backup certain files, even if you delete the plist and start over when you start Steam it will again tell the plist to exclude those files. (I don't know that Steam is doing this... I'm just using it as an example.)

I also thought about installing the new Mountain Lion update and hope it would magically cure the problem, but without a backup that's probably a risk.

I agree with you.

By the way, if I have corrupted files, how can I detect them? I'd expect the Disk Utility tool took care of this, but it seems it doesn't, at least not completely.

Disk Util would find corrupted folder structures and partition errors, but it does not check individual files. For example if you had a corrupted Excel document, Disk Util would not tell you that.

The way to narrow this down will be to exclude some folder that showed the error (like that Steam folder) then do another TM backup and look in the logs. If we see new file/folders with errors that tells us it is likely not corrupt files but more general data corruption from perhaps a bad drive or cables.

Do you happen to have a second external drive to use as a test to see if the problem persists?

 

[NunoBerny]

I'm back.

Yes, but if it is the #3 option, and say Steam tells TM not to backup certain files, even if you delete the plist and start over when you start Steam it will again tell the plist to exclude those files. (I don't know that Steam is doing this... I'm just using it as an example.)

I see... but I didn't run Steam in-between my (failed) TM operations.
My Steam folder is quite big, it could be it (but still the size of the excluded files is somewhat bigger than that too)

I'll exclude it on my next backup attempt.

Disk Util would find corrupted folder structures and partition errors, but it does not check individual files. For example if you had a corrupted Excel document, Disk Util would not tell you that.

The way to narrow this down will be to exclude some folder that showed the error (like that Steam folder) then do another TM backup and look in the logs. If we see new file/folders with errors that tells us it is likely not corrupt files but more general data corruption from perhaps a bad drive or cables.

Do you happen to have a second external drive to use as a test to see if the problem persists?

I do have a second USB3 drive, that was used for my TM backups before I bought the new one and is now used as "normal" storage".
Given that it always worked fine with TM, I assumed the new WD USB3 drive was faulty, but the new one seems to display the same problems.

If all else fails, I can do a copy of my system HD (to the USB3 drive) and then re-install Mountain Lion.
All my files would be secure, but given the complexity of the way files are handled by OSX, I still may have problems.
Example: I copy file or program X, but it needs hundreds of other "auxiliary" files that will NOT be in the new OSX install.

 

[Weaselboy]

I see... but I didn't run Steam in-between my (failed) TM operations.
My Steam folder is quite big, it could be it (but still the size of the excluded files is somewhat bigger than that too)

I'll exclude it on my next backup attempt.

I believe Steam runs a launch agent on OS X to auto check for updates, and I wonder if that would add the exclusions upon system start?

You might try what I posted earlier (below) and peak at the plist file to see exactly what has been excluded.

To see what #3 is up to you can use Quicklook to peek in the file at /Library/Preferences/com.apple.TimeMachine.plist and look for entries under <key>Excludebypath<key>. Say a developer makes an app that creates a bunch of temp files that don't need to be backed up, the dev. can have his app add this entry to the TM plist to prevent needless backup of the temp files.

 

[NunoBerny]

I believe Steam runs a launch agent on OS X to auto check for updates, and I wonder if that would add the exclusions upon system start?

You might try what I posted earlier (below) and peak at the plist file to see exactly what has been excluded.

The only exclusions on my current prefs file are those I manually entered (Steam).
The backup went well, but it's still only a fraction of the whole main HD.

One thing that occurred to me:
I remember I once, long time ago, tried to re-install the system through TM (the idea was trying to revert to a few days before, since an update was giving me problems on a specific program). I wasn't successful... ;(
At the time I didn't think much of it, the system continued working as it should.
Could it have been that I (mistakenly) copied duplicate files to my main HD?!?
This could be why the backup is smaller than it should be, caused by TM identifying some files as "copies", so to speak.

Still, for the backup to continuously fail other problems may be present.

 

[Weaselboy]

The only exclusions on my current prefs file are those I manually entered (Steam).
The backup went well, but it's still only a fraction of the whole main HD.

One thing that occurred to me:
I remember I once, long time ago, tried to re-install the system through TM (the idea was trying to revert to a few days before, since an update was giving me problems on a specific program). I wasn't successful... ;(
At the time I didn't think much of it, the system continued working as it should.
Could it have been that I (mistakenly) copied duplicate files to my main HD?!?
This could be why the backup is smaller than it should be, caused by TM identifying some files as "copies", so to speak.

Still, for the backup to continuously fail other problems may be present.

I don't think that would create duplicates. If you do a command-i on Macintosh HD that would show total size and it would be accurate.

What do you see now in the logs with the Steam stuff excluded?

 

[NunoBerny]

I don't think that would create duplicates. If you do a command-i on Macintosh HD that would show total size and it would be accurate.

What do you see now in the logs with the Steam stuff excluded?

The backup reported by TM prefs is the same as my HD size. Only the backup done is smaller.


The successful backup has a few entries, but the one that stands out is this:

"7/27/13 9:18:22.470 PM com.apple.backupd[843]: Backup content size: 385.72 GB excluded items size: 270.11 GB for volume Macintosh HD"

There are still exclusions that were definitely not ordered by me!
I wonder where that info is being kept!