Time Machine Backup Speed

Discussion in 'macOS' started by downingp, Dec 26, 2012.

  1. downingp macrumors 6502a

    Joined:
    Jun 26, 2006
    #1
    I recently received an error from my Time Machine backup saying that I needed to reformat and erase my external hard drive before Time Machine could back up to the hard drive.
    I followed the instruction and wiped my backup clean and reformatted. I then set up Time Machine backup again using the same hard drive but was curious about the estimated completion time. I have about 500 gb of data to backup and am using USB 2.0. Time Machine is estimating that it is going to take "about 10 days" to backup!!
    That can't be right can it?? It has been backing up with a few hours now and I am only at 7 gb currently.
    Is there something wrong with my hard drive or does this sound about right??
     
  2. switon macrumors 6502a

    Joined:
    Sep 10, 2012
    #2
    RE: TM backup speeds...

    Hi downingp,

    We can estimate a lower bound on the time it should take by using the maximum speed of a USB2.0 connection. This maximum communication speed is 480Mb/s (=60MB/s). Of course, the actual data transfer rate will be less than this because of overhead, etc., but we will be computing a lower bound on the time so we'll assume this maximum rate of 60MB/s.

    Since you have 500GB to back up which is equal to 500000MB, then the time to backup this is 500000MB/(60MB/s)=8333.3 seconds=138.9 minutes or roughly 2 hours and 20 minutes. Mind you, this is the absolute minimum time for your backup. But this is so much lower than your "10 days estimate" that let's compute how fast you are backing up if indeed it takes 10 days=240 hours=14400 minutes=864000 seconds. Your data transfer rate would then be 500000MB/864000s=0.5787MB/s=578.7kB/s. This is considerably less than your theoretical maximum speed of 60MB/s, being over 100 times slower.

    So, what is going on here? Well, there could be two possibilities.

    First of all, when backups initially start they typically have to transfer many many tiny files, and because of the per file overhead, these tiny files are transferred at much reduced data rates. The "estimated backup times" are initially based upon the initially transferred tiny files, and so the time is way overestimated. Usually, even though the backup time is estimated at 10 days, as the backup continues and larger files are copied to the backup drive, the estimated times will decrease drastically. Backing up 500 GB over USB2.0 won't happen in the theoretically lower bound of 2h20m, but I would suspect it to happen in around 2 to 5 times this interval, so from 5 to 12 hours. If it is not done within a day, then I'd begin to worry.

    Now let me ask, did you "erase" the free space on your drive when you reformatted it? Simply reformatting your problem drive may not have been enough. Why? Well, when a disk drive has read/write errors, it automatically moves the "blocks" having the R/W errors to unused saved "blocks" whose purpose is exactly this, to serve as a backup for blocks experiencing errors. The blocks having errors are then abandoned and the "extra" blocks are now used for all future storage. On a disk drive experiencing R/W errors, an "erase of the free space" will go through all of the blocks on the newly formatted drive and mark any that are having problems so that these marked blocks will not be used by the drive for its data storage. I'm not certain exactly what the Mac OS's Disk Utility formatting does, but it may very well require the "erase of free space" in order to mark all blocks having errors that should not be used. Now if you did not "erase free space" when you reformatted your USB2.0 HDD, then TM may be running into some "bad" blocks when it is backing up and any R/W errors may have a time-out of up to 7 seconds before the block is transferred to a new unused block. All of this can slow the TM backup. (I also dont' know if TM actually tests the disk blocks before it backs up to them, it could but I sort of doubt it does, or the drive itself through ERC may be delaying the backup or the CoreStorage extensions of the OS may delay the backup when R/W errors are encountered.)

    If you find that your USB2.0 HDD is having a lot of R/W errors, then it would be wise to abandon that drive as a backup. You risk loosing your backup if you don't. Personally, I backup to enterprise-class HDDs instead of commercial grade HDDs, as these have better warrantees and longer MTBFs, and thus hopefully they are better made with better tolerances and so should last longer. They also have better vibration isolation so are less prone to vibration induced troubles.

    Regards,
    Switon
     
  3. downingp thread starter macrumors 6502a

    Joined:
    Jun 26, 2006
    #3
    Thank you for that very informative explanation. I did use disk utility to erase and reformat the drive into Mac OS (Journaled).
    I will take your advice and let Time Machine run for a day or so to see if things pickup as a first step and then take it from there.

    Thanks again.
     
  4. switon macrumors 6502a

    Joined:
    Sep 10, 2012
    #4
    Hi,

    There is a difference between simply "erasing" and "erasing free space" on a drive. The "erasing free space", which often takes hours to perform, is generally the method that tests every block on the drive. Personally, if I had a drive that reported a R/W error and that I wanted to continue to use as a backup disk, I'd reformat it and do an "erase free space" on it before using it for TM backups.

    ...just a suggestion...

    Regards,
    Switon
     
  5. downingp thread starter macrumors 6502a

    Joined:
    Jun 26, 2006
    #5
    Thanks again. Can I do an "erase free space" within disk utility?
    Luckily I have backups in addition to time machine so all would not necessarily be a complete loss if my time machine backup bombed out on me. Nonetheless, it may be beneficial to get a new hard drive.
     
  6. switon, Dec 27, 2012
    Last edited: Dec 28, 2012

    switon macrumors 6502a

    Joined:
    Sep 10, 2012
    #6
    RE: Erasing...and backups...

    Hi downingp,

    Well, as I said, my personal opinion is to replace the drive you use for backups, use the one that has already experienced a non-recoverable error as your day-to-day working drive, use the new better drive for your TM backup drive --- just my opinion, of course.


    Regards,
    Switon
     
  7. Bear macrumors G3

    Joined:
    Jul 23, 2002
    Location:
    Sol III - Terra
    #7
    The estimate sounds really off to me. Is the drive directly connected to the Mac or is it through a USB HUB?

    If it's through a USB hub, is the hub actually USB 2.0?

    Have you tried using a different USB cable? It could be a drive going bad or it could be a bad cable.
     
  8. downingp thread starter macrumors 6502a

    Joined:
    Jun 26, 2006
    #8
    I just swapped out my time machine backup with another, older external hard drive just to test things out. It is another WD drive with firewire 800. I started an initial time machine backup and it finished in less than 8 hours (went to bed). Can I assume my original backup hard drive has gone bad?
    Any tests or programs I can use to analyze and/or fix the drive?
     
  9. switon macrumors 6502a

    Joined:
    Sep 10, 2012
    #9
    RE: checking a HDD...

    Hi,

    There are any number of tools, including Mac OS X system tools for checking your HDD. Oftentimes the vendor also has tools...and there are numerous 3rd party tools for checking and "fixing" HDDs. Realize that these tools are mostly just switching to the "spare" blocks on the HDD, and if you HDD is experiencing other troubles then it may still fail even after "fixing" it with one of these tools.

    Good luck,
    Switon

    P.S. Firewire 800 is roughly, in practice, twice the speed of USB2.0, so backups of the same material should happen roughly twice as fast on Firewire 800 over USB2.0. Many if not most modern HDDs are capable of faster communications than either Firewire 800 or USB2.0.
     
  10. downingp thread starter macrumors 6502a

    Joined:
    Jun 26, 2006
    #10
    I understand that Firewire 800 was going to be faster than USB 2.0 but I wanted to get a backup and wanted to see the speed difference
    Also, I have run a bunch of tests on the drive and they all check out fine. Do you still think there is a problem with the drive? Should I do another erase and this time do an "erase free space?"
     
  11. switon macrumors 6502a

    Joined:
    Sep 10, 2012
    #11
    I would definitely do an erase on the free space before I used the drive as a backup drive. (Actually, I'd use Linux tools, but that is me. I assume Apple's tools will perform similar checks and marking of bad blocks as the Linux tools.)

    Regards,
    Switon

    P.S. As I mentioned, the estimated times at the start of a TM backup are often wildly overestimated because of the initial copy of many tiny files. Did you actually complete the USB2.0 backup? And, if so, how much time did it take?
     
  12. downingp thread starter macrumors 6502a

    Joined:
    Jun 26, 2006
    #12
    I never did complete the USB2.0 backup. After two days it was still estimating 9 days remaining. I did, however, test the USB2.0 drive with transferring just a small 2 GB file and it said the file would take an hour to complete. I tried with both the USB2.0 interface as well as firewire 4.0 with the same results. Could the hard drive actually still be good and my enclosure be bad or is it definitely a failing hard drive?
     
  13. switon macrumors 6502a

    Joined:
    Sep 10, 2012
    #13
    Yes, the HDD itself (physical platters) could still be good with the interface bad...this is definitely a possibility. Transferring 2GB over USB2.0 should take nowhere near an hour, so somewhere you are having troubles. Since you tried both FW and USB (using different cables, etc.) with similar slow rates, then this points to a drive problem. Can you switch enclosures? Do you have another USB2.0 drive to check that you get reasonable transfer speeds?

    Switon
     
  14. switon macrumors 6502a

    Joined:
    Sep 10, 2012
    #14
    RE: Erase free space...

    Hi downingp,

    I just looked up and apparently the Disk Utility.app does indeed remap any blocks that experienced errors while doing an Erase of Free Space. This is what I suspected, and it appears that this is true. So, to reiterate, I would do an Erase of free space on your drive before using it. This can take a relatively long time, as it writes zeroes to your whole drive and reads them, so be patient.

    Switon
     
  15. downingp thread starter macrumors 6502a

    Joined:
    Jun 26, 2006
    #15
    Thanks for all of your help
     
  16. switon macrumors 6502a

    Joined:
    Sep 10, 2012
    #16
    You're welcome and good luck with your drive...

    Switon
     

Share This Page