Issue with moving Time Machine backup to new external drive

[DevyV]

Hi,

I work on macOS Mojave 10.14.6 with latest Security Update 2020-004.

I am following this article by Apple to move my Time Machine backup to a new USB disk:
https://support.apple.com/en-us/HT202380

My old Time Machine backup is on a 500 GB external HDD called "macOS Time Machine":

As you see above, 296,32 GB used on this drive.

Here it is important to know that besides the Time Machine backup ("Backups.backupdb" folder) I have 2 other folders on the drive taking up 73,87 - 69,0 = 142,93 GB space:

So 296,32 - 73,87 - 69,06 GB = 152,49 GB is the size of Time Machine backup ("Backups.backupdb" folder).

So far I created a new 665,92 GB partition on the new external HDD:

Then dragged+dropped the "Backups.backupdb" folder to this new drive's new partition.

After a night of copying now I see this:

The copy window shows me that macOS copied 414,5 GB to the new drive already, but Disk Utility shows the used space only 344,58 GB. How can it be? (yes, I refreshed the Disk Utility window)

Also, on the first screenshot above I see that Drive Utility reports 296,32 GB is used on the old drive, so how could macOS have copied already 414,5 GB (and still copying) even considering that we know I have 2 other folders on the source drive that I do not copy over, so as we calculated: Time Machine folder needs to be 152,49 GB only that is to be copied?

Looking at the partition information is also pretty interesting:

Source partition (left side) has 3.163.830 files while the target (right side) has 7.208.732 files and growing!

How could this happen? Where are those 4 million more files come from?!

Also, the copy windows looks pretty weird showing "Copy 0 items to..." and not being able to calculate the size of the data to be copied: the "414,50 GB of 414,5 GB" numbers keep growing in parallel and the "About 5 seconds" keep stay forever "5 seconds".


What is wrong with this simple copy process?
How can macOS report so false numbers for a simple folder copy operation?
How can Drive Utility report bad numbers?

Thanks for any help in advance!
DevyV

 

[chabig]

I see your drives are both formatted case-sensitive. That's unusual. Also, instead of showing us windows from Disk Utility, can you show us what you see in Finder? You should only see Backups.backupdb.

I suspect that you copied Backups.backupdb to the new drive, and then set up Time Machine again so that it's making a second copy.

 

[DevyV]

Thank you chabig for looking into my issue!

As far as I remember, the source Time Machine partition was created by Time Machine itself when I ran it for the very first time and all it asked me if I wanted to encrypt the volume or not - where I chose 'encrypt'. So I did not chose 'case sensitive' option myself.

In many guides I read to create a 'Journaled' filesystem and really nowhere I read about the 'case-sensitive' option - but now that is already done on the source drive.

When I created the target partition I first did not choose to be case-sensitive but when I started the copy process macOS did not do the copy asking me to format the partition to case-sensitive which I did so with Disk Utility.

That's why I have case-sensitive partitions.

On the other hand, I just started to copy Backups.backupdb to the new partition - I did nothing else, even more because copying still has not finished and I did not got to the point to change the Time Machine destination disk in its settings, so Time Machine is still set to use the old (source) drive (and I never had auto backup enabled so it should never start any backup by itself):

I just now stopped the never ending copy process, unmounted and remounted the external drive.

Finder shows the source drive as:

and the target drive as:

 

[chabig]

Keep in mind that I am speculating. Your target drive shows just the one folder, but it must have more files in it than the original. I don't know why.

What I would do is reformat the new Time Machine volume to Apple's preferred format, Mac OS Extended (Journaled), and let Time Machine start over from scratch. Set the original Time Machine drive aside as an older backup, just in case. While you might think you're losing years of backups, all you're really giving up is the ability to go back in time for older versions of files. The main value of Time Machine is that it protects you if the primary drive fails, and yours hasn't. That's why it's ok to start fresh.

 

[DevyV]

Yeah. After all it is just unbelievable that macOS cannot do such a simple copy operation

I am quite new to MacBook Pro and macOS but I am an IT engineer and programmer for 30 years and anything I saw on screen today makes me asking "so why are there so many Apple users on this Globe?" Even the easiest basic task like copying files between drives requires so much extra effort from me to complete (

Anyway, I cancelled the native copy and downloaded SuperDuper! to see if that makes a difference.
And that does:

As you see, it correctly reported the number of files to be copied to the new drive - at least.
I do not know if it really will do the job correctly at the end of the day but I keep my eyes open...

(Btw, it is also a mystery to me why copying between the two USB 3.1 SATA III drives go 4,19 MB/s slow. These drives should be 600 MB/s fast which is about 142x faster than how fast my MacBook Pro 2019 performs. Wow! Great job, Apple!)

 

[DevyV]

For to help others, my solution:

SuperDuper! simply copied these files over to the new partition after erasing and now I have exactly the same files on the source and target drives as far as I can measure for first sight.
Of course, SD not only copied the Backups.backupdb folder but the 2 other folders as well that I can simply delete now.
It took about 15 hours for my 296,32 GB (from USB-C to USB-C disk).

But not to let us go sleeping well with aforementioned mysteries only, now when restarted my MacBook today and plugged in the New Time Machine drive only, it looks like macOS recognised it somehow and automatically set itself up to use it as the Time Machine volume - despite all guides (incl. Apple's) that say: "After copying the old backup file to the new disk, select the new disk in Time Machine preferences"

So the last mystery for those who got this far with Apple: strangely now the new Time Machine volume is reported as

1. "Case-sensitive, Journaled" (in Finder)
2. "Encrypted" (in Time Machine)
3. "Encrypted, Case Sensitive" (in Time Machine)

Last two are both within the same application! A total mix-up! Which one should you vote for now?
Looks like macOS may not even know about itself the basic things... just keep running somehow!!! Great job, Apple!

+ make sure to read a helpful advice from John

 

[Phantom Gremlin]

I went thru this a few years ago. It's a big hassle.

IIRC this is the article I found that was what I wound up doing: https://posts.boy.sh/to-move-your-time-machine-backup-to-another-disk-use-disk-utility

This was before the "update" added to that article. I didn't use Finder, I used Disk Utility. From other articles, if I understand correctly, Disk Utility actually uses Apple Software Restore (asr) to do the dirty work.

Also, that was with (probably) El Capitan and was with HFS+ spinning drives. So YMMV with the latest OS, APFS, and SSD.

The problem with stuff like SuperDuper is it won't retain the time machine history. At least I think so, I haven't checked it out.

I can confirm that, when I did whatever it was I did, the Time Machine history was preserved, and there was no increase in overall space required. At the time I had a 2 TB drive for TM. It was almost full. I copied to a 4 TB drive and wound up with 2 TB free.

Also, I don't know if the "case sensitive" stuff will have any bearing on this method.

 

[Fishrrman]

I'd switch to SuperDuper or CarbonCopyCloner.
You're having problems with tm already. Why keep using it?

 

[Phantom Gremlin]

You're having problems with tm already. Why keep using it?

Time machine can do things that the others can't.

Time Machine has special access to file system primitives which allow it to utilize disk space more efficiently than SuperDuper or CarbonCopyCloner can. Specifically, IIRC, TM can do what unix calls "hard links" to directories, not just to files. This is normally not allowed (probably because of the danger of circular links).

These hard links allow quick copies to be made of entire directories, not just files. This makes it very time and space efficient for TM to maintain a large number of old snapshots.

I'm not very familiar with SuperDuper; I use CCC and know that it offers a history feature which preserves multiple versions of files. But without the special file system operations available to TM, CCC can't be nearly as efficient in doing this.

 

[Fishrrman]

Again, my reply is:
The OP is trying to use TM and having problems.
If he switches to CCC or SD, the problems will end.