Copying Time Machine folder to another HDD

[nelly22]

I have 10.14.2 and i unchecked "Back Up Automatically" in Time Machine preferences and now try to copy small about 10GB Backups.backupdb folder from HDD to another another HFS+ drive but after hours of copying, it says "Copying 0 items to "Time Machine" 12,22 GB of 12,22 GB - About 5 seconds"

This continues for hours and both 12,22 numbers slowly grow.

Is this normal?

 

[JRBuchanan]

I have 10.14.2 and i unchecked "Back Up Automatically" in Time Machine preferences and now try to copy small about 10GB Backups.backupdb folder from HDD to another another HFS+ drive but after hours of copying, it says "Copying 0 items to "Time Machine" 12,22 GB of 12,22 GB - About 5 seconds"

This continues for hours and both 12,22 numbers slowly grow.

Is this normal?

Yes, I believe so. I just migrated my Time Machine backup from a 1TB disk to a new 2TB disk and saw the same behavior. It took about 72 hours to completely copy the backup.

 

[BrianBaughn]

I think it would be better to use Disk Utility's "Restore" function for this purpose. The TM backup folders contain mostly "hard links" of files which take up little space...but when you "copy" them you're copying the full files.

 

[JRBuchanan]

I think it would be better to use Disk Utility's "Restore" function for this purpose. The TM backup folders contain mostly "hard links" of files which take up little space...but when you "copy" them you're copying the full files.

I wish I had known about that process. Instead I googled how to do this and followed the instructions I found at the link:

https://support.apple.com/en-us/HT202380

 

[nelly22]

I think it would be better to use Disk Utility's "Restore" function for this purpose. The TM backup folders contain mostly "hard links" of files which take up little space...but when you "copy" them you're copying the full files.

Does that work if Backups.backupdb is folder not disk image?

I think many these "use disk utility to backup time machine" instructions talk about partitions.

 

[Weaselboy]

Does that work if Backups.backupdb is folder not disk image?

I think many these "use disk utility to backup time machine" instructions talk about partitions.

Yes... it will work.

When you use Disk Util to "restore" one volume to another, you are basically cloning that volume.

 

[mlody]

Yes... it will work.

When you use Disk Util to "restore" one volume to another, you are basically cloning that volume.

I tried that and it didn't work.

I am doing normal backup (per apple white paper) and it is taking forever. The source disk is only 250 GB and of which 200 GB is Time Machine backup. The new drive is 1 TB.

Now the copy message says:

"Copying 0 items to 'backup'"
727.20 GB of 727.2 GB - About 5 seconds.

How the heck my 200 GB time machine backup turned into 727 GB?

The strange thing, this number keeps growing.

Is this normal?

 

[Honza1]

It is normal. Copying TM disks is difficult. TM creates hard links (as stated already above) and any file (or its parts!) is there only once. But it may be linked many, many times into different places in the folder tree. That way TM is very very space efficient, effectively storing many many times more of similar "copies" of the complete drive than its space. At cost of speed and reliability (one file fails, all copies are gone). And the inability to duplicate the disk. Basically, you are creating a new copy of each file every time the original is linked, Finder cannot make hard links.
Something like current APFS does - a file is stored once but potentially linked as many times as needed. That is why file duplication/copying on the same drive in APFS is instantaneous.
Personally, I gave up on moving TM disks many years ago. Pull the old one and store as backup, just in case, start a new TM disk. 200GB disk today costs nothing, you can get USB sticks of that size for $40 or so. Not worth the bother. And if you need occasionally an old file, simply pull the disk out, connect and use Finder to find file you need.

 

[treekram]

I tried that and it didn't work.

I am doing normal backup (per apple white paper) and it is taking forever. The source disk is only 250 GB and of which 200 GB is Time Machine backup. The new drive is 1 TB.

Now the copy message says:

"Copying 0 items to 'backup'"
727.20 GB of 727.2 GB - About 5 seconds.

How the heck my 200 GB time machine backup turned into 727 GB?

The strange thing, this number keeps growing.

Is this normal?

There is something wrong with the 10.14.4 (and perhaps earlier Mojave updates) Time Machine copy function. I followed the instructions in the link in post #4 above. I've been working for days (obviously not continuously, but it takes hours to complete a copy or to determine that a copy is failing) trying to figure this out. I'm not finished yet but this is what I've noticed: If I do the copy with 10.14.4, it will go into a loop as described in the original post. In one case, a backup that was about 680GB grew to over 1TB when I left it overnight and it was still going. Obviously, something is wrong there. If I do the same copy in High Sierra (10.13.6 with Security Update 2019-002 - 17G6030), it doesn't have this problem - the copy isn't larger than the original. Right now, I am checking this backup which was copied using High Sierra (using tmutil compare) and I'm in the middle of checking the 2nd of 3 backup directories that I'm testing out but so far it looks good. In the cases where I did the copy with 10.14.4, even though the size of the backups was much larger than the original, it had fewer backup dates on it, so I think it's not following the file links that Time Machine uses properly in 10.14.4.

 

[treekram]

As mentioned in my previous post, it's pretty obvious that the Time Machine backup copy function as outlined in:
https://support.apple.com/en-us/HT202380
isn't working properly in Mojave 10.14.2 (and it looks like, at least back to 10.14.2).

I'm pretty confident that using High Sierra (I used 10.13.6 with Security Update 2019-002 - 17G6030) will properly copy a Time Machine from one disk to another. I say "pretty confident" because of the uncertainty surrounding exactly how one compares two backups on two disks. The "tmutil compare" command-line program without any options will only compare metadata. The "tmutil verifyChecksums" doesn't appear to have the functionality to compare two disks, just the checksums and the file within a backup directory. You would think "tmutil compare" would compare file checksums (available since El Capitan) but the documentation is mute on that point. That leaves "tmutil compare -d", which I used. But there were times when it couldn't get started. But it worked enough to the point where I believe it was doing it's job and I compared 3 backup dates successfully (the first Sierra backup, last Sierra backup and last Mojave backup on my disk). My "tmutil compare -d" travails would just take up too much space here so I'll leave it at that.

I used High Sierra on a different computer than the computer where the backup was created. It didn't cause a problem for me (my user ID and name are the same on both computers) but I'm not sure what will happen if the user ID of the creator of the backup on the original computer differs from the user ID trying to execute the backup on another computer. Somebody more knowledgeable in this area may be able to provide more insight into this.

So if you can't go the High Sierra route, what are the options? CCC says it can't do it (TM too proprietary). People have said Super Duper can but if you go this route, check to see if the people reporting this checked to see if the disk space used hasn't expanded and if some sort of comparison has been done on the two disks. I tried doing a byte-by-byte copy to another disk and then expanding the partition but there were problems in expanding the partition (this worked for a test I did on a non-TM backup disk). So I think the alternative would be to leave the old disk as it as and start a new Time Machine backup disk.

For me, after this episode, I'm going to stop using Time Machine to backup my primary computer. I think I'm going to use CCC on a APFS SSD using the snapshot feature built into APFS. I'm looking at the Samsung 860 QVO which is $108 for 1TB on Amazon. I wouldn't use the QVO as a system disk, but this seems to be a good alternative for this type of backup. CCC doesn't recommend doing this type of APFS snapshot backup on HDD's and just about all my media is on HDD's so my system drive doesn't take up that much space and 1TB will last me awhile. I currently have 2 SSD clones of my system disk (plus an offsite clone) so I'm well covered even as my last TM backup ages until I get around to making this work. I'll still keep my archival TM disks and at some point I'll decide to do with my other Macs - I have both clones and TM backups for these.

(CCC does have a safety-net feature which is an incremental backup that can be used on HDD's but it's pretty apparent to me at least that use of the snapshot built into the OS file system to keep incremental backups is going to be the way to go in the future.)

 

[mlody]

As mentioned in my previous post, it's pretty obvious that the Time Machine backup copy function as outlined in:
https://support.apple.com/en-us/HT202380
isn't working properly in Mojave 10.14.2 (and it looks like, at least back to 10.14.2).

I'm pretty confident that using High Sierra (I used 10.13.6 with Security Update 2019-002 - 17G6030) will properly copy a Time Machine from one disk to another. I say "pretty confident" because of the uncertainty surrounding exactly how one compares two backups on two disks. The "tmutil compare" command-line program without any options will only compare metadata. The "tmutil verifyChecksums" doesn't appear to have the functionality to compare two disks, just the checksums and the file within a backup directory. You would think "tmutil compare" would compare file checksums (available since El Capitan) but the documentation is mute on that point. That leaves "tmutil compare -d", which I used. But there were times when it couldn't get started. But it worked enough to the point where I believe it was doing it's job and I compared 3 backup dates successfully (the first Sierra backup, last Sierra backup and last Mojave backup on my disk). My "tmutil compare -d" travails would just take up too much space here so I'll leave it at that.

I used High Sierra on a different computer than the computer where the backup was created. It didn't cause a problem for me (my user ID and name are the same on both computers) but I'm not sure what will happen if the user ID of the creator of the backup on the original computer differs from the user ID trying to execute the backup on another computer. Somebody more knowledgeable in this area may be able to provide more insight into this.

So if you can't go the High Sierra route, what are the options? CCC says it can't do it (TM too proprietary). People have said Super Duper can but if you go this route, check to see if the people reporting this checked to see if the disk space used hasn't expanded and if some sort of comparison has been done on the two disks. I tried doing a byte-by-byte copy to another disk and then expanding the partition but there were problems in expanding the partition (this worked for a test I did on a non-TM backup disk). So I think the alternative would be to leave the old disk as it as and start a new Time Machine backup disk.

For me, after this episode, I'm going to stop using Time Machine to backup my primary computer. I think I'm going to use CCC on a APFS SSD using the snapshot feature built into APFS. I'm looking at the Samsung 860 QVO which is $108 for 1TB on Amazon. I wouldn't use the QVO as a system disk, but this seems to be a good alternative for this type of backup. CCC doesn't recommend doing this type of APFS snapshot backup on HDD's and just about all my media is on HDD's so my system drive doesn't take up that much space and 1TB will last me awhile. I currently have 2 SSD clones of my system disk (plus an offsite clone) so I'm well covered even as my last TM backup ages until I get around to making this work. I'll still keep my archival TM disks and at some point I'll decide to do with my other Macs - I have both clones and TM backups for these.

(CCC does have a safety-net feature which is an incremental backup that can be used on HDD's but it's pretty apparent to me at least that use of the snapshot built into the OS file system to keep incremental backups is going to be the way to go in the future.)

Thank you for the extensive write up. I actually decided to ditch the old TM backup drive and started fresh on the new HDD. The SSD drive with the legacy backup I will be repurposing for a different project, but I got about a month before I will need the drive, so this gives me enough of the cushion to use the new backup drive and perhaps have up to a month of roll back capabilities - by then if I do not need to use it, I wont need it anymore, so all good.

 

[treekram]

Thank you for the extensive write up. I actually decided to ditch the old TM backup drive and started fresh on the new HDD. The SSD drive with the legacy backup I will be repurposing for a different project, but I got about a month before I will need the drive, so this gives me enough of the cushion to use the new backup drive and perhaps have up to a month of roll back capabilities - by then if I do not need to use it, I wont need it anymore, so all good.

I was trying to copy my TM backup from a 1TB disk to a 2TB disk when I encountered this problem. I was thinking about starting a new thread but saw this thread and latched on to it. I suspect that most people will start a new backup disk if they encounter this problem but I think that this is basic functionality that should work. I mean, what if you have media on a 2TB disk and it's almost full and want to move it to a 4TB disk. "Oh I'm sorry, you can't do that, you need another 2 TB disk or if you plan to even have more media in the future, consider getting a 8TB or 12TB disk or maybe a RAID enclosure".

I did try earlier this evening to create a folder image in Disk Utility which is what people were proposing earlier? For my backup folder, which is about 660GB, it took 4 hours, over 16GB of RAM and ended with "Operation failed with status 45: Operation not supported". This was before any bytes were actually written to the image file. I'm not exactly sure how much the high-water RAM point was but it was probably in the mid-16GB range. Clearly not practical even if it did work as many Macs don't have more than 16GB of RAM (mine does have 32GB so I guess 32GB does come in handy sometimes). Now that I think of it, maybe it threw an error because it did reach 16GB.

 

[jimthing]

Is it me, or is Time Machine just that: a machine that sucks all your time trying to get it to remain working!?

AFAICT, over the years it has become more and more unreliable as a backup application, especially in the age of TB's of externally stored data, that the help section doesn't even mention about!
For example, here's the current so-called "macOS User Guide" for Mojave, where it mentions ABSOLUTELY NOTHING about backing-up any external drives connected to a Mac, whatsoever:
https://support.apple.com/en-gb/guide/mac-help/back-up-files-mh35860/mac
(sure, it says to use an external as your backup location, but nothing about one being a source!)

And like 99% of users, we're not tech savvy enough to get into command-line stuff either (nor likely interested in doing so; as us 'norms' really shouldn't have to resort to using it, Apple!), just in order to make using and moving TM backups.

Reminds me of that message many of us have had saying words to the affect "Cannot verify this backup. Click OK to start a new Time Machine backup" or similar
– i.e. you're backup is completely screwed (again, you can try various things, but the chance of recovery success following this error message is almost nil).

(CCC does have a safety-net feature which is an incremental backup that can be used on HDD's but it's pretty apparent to me at least that use of the snapshot built into the OS file system to keep incremental backups is going to be the way to go in the future.)

Can you explain the bolded point you're trying to make further. How exactly can users make use of this functionality currently?
AFAIK, it's basically only useful for file version changes, rather than a backup??
Is it even documented properly anywhere...

 

[treekram]

(CCC does have a safety-net feature which is an incremental backup that can be used on HDD's but it's pretty apparent to me at least that use of the snapshot built into the OS file system to keep incremental backups is going to be the way to go in the future.)

Can you explain the bolded point you're trying to make further. How exactly can users make use of this functionality currently?
AFAIK, it's basically only useful for file version changes, rather than a backup??
Is it even documented properly anywhere...

Basic history lesson - with HFS+ and many older file systems (I haven't kept up with what's going in this regard on in Windows and Linux), the way you figure out what files have changed between two sources is to go through each directory listing and compare date/time. With APFS, it stores data on a file's state relative to the snapshots it has - so in theory at least (APFS is Apple-proprietary), it's much faster and more reliable in terms what files actually need to be backed up. There's no doubt that this approach is superior than what has been done in the past. Obviously this is only the case if it works correctly.

In the context of APFS, a snapshot is a list of all of the files on a source [disk/directory/etc.] at a particular point in time. If the snapshot exists and you "delete" a file in the snapshot, it's not actually deleted - your typical file listing shows it as deleted, but APFS keeps a copy until you delete the snapshot. So any snapshot is a full listing of what you have on the source at a particular point in time and a behind-the-scenes hook to those files. So the snapshot can be used to do a full restore for what existed at the time of the snapshot. When you delete a snapshot, you delete the listing and any files that were unique to that snapshot. That's why you see posts on these forums here where deleting snapshots freed up a lot of space - the listing itself isn't that large, but files that only existed in specific snapshots were deleted.

So yes, APFS snapshots do incremental (file version changes) backups if you keep snapshots and do them often enough to be useful but it can be used as a backup - IF you use APFS on your backup disk (a disk different than your source disk). So to do this, your backup disk needs to be a SSD at the moment because it seems there are still kinks to be worked out in using this on a HDD. Also, to make use of this functionality in a easy way, you need something like CCC which can manage your snapshots and integrate the copying of the files and creating the snapshot on the backup disk. Apple doesn't have such a user-friendly tool at the moment. I'm sure you can do something like that without using CCC, but it won't be as easy to use.

So I think for CCC, you create a backup task - then enable snapshots on the backup disk and turn the safety net on. I've ordered my backup SSD and when I have something to report in using this, I might start a thread here or in the macOS forum (since you can do this in both Mojave and High Sierra).

UPDATE: In CCC, SafetyNet snapshots are used to help prevent mistakes. "Backup snapshots" are used for backup events.

 

[crjackson2134]

TM works great... Until it doesn’t!

For me Time Machine was incorrectly named. It should be called Time Bomb instead. I ditched it, bought CCC and that works great for me.

 

[mikzn]

TM works great... Until it doesn’t!

For me Time Machine was incorrectly named. It should be called Time Bomb instead. I ditched it, bought CCC and that works great for me.

I’ll second that comment

Time Machine is a PITA

carbon copy cloner is awesome, works great, easy to use

 

[jimthing]

Basic history lesson - with HFS+...

...So I think for CCC, you create a backup task - then enable snapshots on the backup disk and turn the safety net on. I've ordered my backup SSD and when I have something to report in using this, I might start a thread here or in the macOS forum (since you can do this in both Mojave and High Sierra).

UPDATE: In CCC, SafetyNet snapshots are used to help prevent mistakes. "Backup snapshots" are used for backup events.

Great explainer. Yeah a thread might be appreciated from many people, so go for it. These things can be quite complicated for the average non-super techie user to understand, so the more info out there the better, especially as Apple often don't really offer much comprehendible info themselves.

To do my small part in the effort, for anyone on the thread...

Here's some backup guidance info, courtesy of Joe Kissell's "Take Control of Backing Up Your Mac: The Online Appendixes" – the free online bit of his guidebook:
https://joeontech.net/extras/buym/online.html

Just skimming the synopsis, and it looks like one with great general guidance as well as specific methodologies for doing backups.

It looks like CCC has more functionality with this, on a simple quick look comparison with SD.

I haven't got or read the full "Take Control Of Backing Up Your Mac (Third Edition)" book myself yet (I'm too busy right now, lol!) but may have to look into doing so, as the latest update to it was in 23.Jan.2019 so it's up to date with Mojave).

As a thanks for his free info, I better give some buying options, depending on your preferred ecosystem. ;-)

International:
£11.50 ($15.00): https://www.takecontrolbooks.com/backing-up

UK:
£12.99: https://books.apple.com/gb/book/take-control-of-backing-up-your-mac-third-edition/id1330213954
£11.51: https://www.amazon.co.uk/Take-Control-Backing-Your-Mac-ebook/dp/B078Z35MTC
(as seemingly always; why is Apple's UK version is 13% more expensive?!)

US:
$14.99: https://books.apple.com/us/book/take-control-of-backing-up-your-mac-third-edition/id1330213954
$14.95: https://www.amazon.com/Take-Control-Backing-Your-Mac-ebook/dp/B078Z35MTC

 

[treekram]

A couple of updates.

So the "Restore" procedure in posts #3 and #6 above involves the following (wasn't sure what the posters meant):
- In the Disk Utility app, select a target volume. The volume will have a name that you assigned to it. The disk name will typically have a manufacturer - model name. Don't select that (if it's visible) - select the volume.
- Right click on "Restore ..."
- Select a volume to restore from
- Press the Restore button.

It turns out that this does work in Mojave - BUT - it looks like you have to have SIP disabled. (Search "macos sip disable" for information on how to do this.) On one of my computers the Restore fails validation immediately with an error code but no explanation but on another with SIP disabled, it works fine. I don't know what else it could be - I can't traverse the TM backup directories in the Terminal app on the same computer where Restore fails but I can on the SIP-disabled computer. [Sorry I didn't opt to take the time to disable SIP on the computer to verify this.] This method works faster than the procedure Apple has for doing a TM copy but I was copying to an unencrypted disk whereas my other test was on an encrypted disk so that may have something to do with it. I did a metadata compare of my latest backup and it checked out fine. But if you do this, you should do additional validation to make sure the copy is good.

Also - IMPORTANT - I didn't see a way to gracefully terminate the Restore if the need arises. I used Terminal to kill Disk Utility but I had to re-erase the target disk after that.

The second update is that I contacted Bombich (CCC) that they said that right now Apple does not have (or hasn't disclosed) a way of copying the APFS snapshot structure to another disk - so you have the same issue of copying backups to another (typically larger) disk in the future should you decide to go the APFS snapshot method of doing backups (as I'm planning to do as mentioned in my post #14). Obviously you can restore an APFS snapshot but if you have 50 snapshots on a disk and want to clone that, right now you can't do that. I'm guessing you can manually restore each snapshot to another APFS disk with CCC where snapshot on the target disk is enabled and then it will be a copy of the source disk but that's quite a pain to do.