Corrupted Files (maybe 4K drive related)

[m85476585]

So in the past 24 hours I've been trying to upgrade my internal hard drive from 320GB to 1TB. It's a little more complicated than usual, because I have a SSD as my primary drive, and the mechanical drive is in a caddy in the optical bay. It's also more difficult because I don't have a built-in super drive any more, and the external DVD drive I normally use stopped reading DVDs (it still reads CDs just fine), so I can't boot from the install disc.

So I put the new drive in an enclosure and tried to use Disk Utility to restore to it from the existing 320 drive. The first time it failed about half way with a vague error that said something like input/output error. I think it was trying to do a file-level copy, and the source drive was still mounted the whole time.

The second time I quit everything and tried again, and it unmounted the source drive and did a block level copy. It took about 6 hours (I let it run overnight), but it appeared to work. In the morning all the data appeared to be there, and I simply had to recreate my sym links to make everything work. However, when I started to use it, I noticed that many of my files were corrupted. I noticed it mainly with image files, but I didn't spend a lot of time trying to test various kinds of files, and I neglected to do a ckecksum on any of the corrupted files. The specific problem was when I tried to open a corrupted image, Preview would say that it does not appear to be a valid file, or something like that.

Rather than risk having the corruption somehow affect data on my SSD (for example, if I tried to open a program with a database that points to corrupt files, the program might decide to delete the database), I tried restoring from my Time Machine backup. That too, appeared to be successful, but it looks like it didn't quite get everything because the number of files and the space used on the drive is now slightly lower. I know Time Machine won't back up caches and temporary files, so maybe this is OK. I plan to try to run a script to compare the contents of the old drive to the new one at some point.

So I'm wondering about the corruption after the block copy, because I haven't seen anything like it before. Only some files were affected, but it looked like a significant portion of them (maybe 20%). Could the problem be caused by the fact that the new drive is an "Advanced Format" drive with 4K sectors? Everything I read says OS K supports those with no problems.

I'm running 10.6.8 on a MBP4,1 with a 60GB SSD in the normal spot, and the new drive in the optical bay with a PATA-SATA conversion caddy. Aside from the new drive, nothing else has changed and I've been using the same hardware setup for at least a year with no problems.

 

[spork183]

not to go off topic, but just curious why you didn't use superduper or carbon copy cloner or one of the other cloning utilities?

 

[m85476585]

not to go off topic, but just curious why you didn't use superduper or carbon copy cloner or one of the other cloning utilities?

Good question. To be honest I didn't even consider them. I didn't see a need to since the functionality I want is built into Disk Utility. I tend to prefer built-in over third-party anyway, unless there's a compelling reason that one of those is better.

 

[GGJstudios]

Good question. To be honest I didn't even consider them. I didn't see a need to since the functionality I want is built into Disk Utility. I tend to prefer built-in over third-party anyway, unless there's a compelling reason that one of those is better.

Carbon Copy Cloner is an excellent tool with a good reputation that enables you to create bootable clones of your drive. You can schedule regular incremental backups that keep that clone current, you can archive modified files, etc. It's highly recommended.

 

[alphaod]

I would use Carbon Copy Cloner.

Block copy doesn't work well for migrating from 512 Byte to 4K sectors. Also doesn't always work for copying from HDD to SSD or vice versa.

 

[m85476585]

In an effort to convince myself that Time Machine restored everything (since there was corruption the first time around with Disk Utility and the number of files and total amount of data on the drive didn't match after a TM restore), I came up with a way to compare two drives.

There's a cool program called FileMerge that comes with Xcode that lets you compare two text documents or supposedly two directories. The directory compare feature doesn't work well, runs slowly, and crashes for large files or directories (or something). So I used du, the built in command to analyze disk usage, to compare the old and new drives. I ran the following commands:

$cd /Volumes/old_drive/Users
$du -a > ~/Desktop/old_files.txt
$cd /Volumes/new_drive/Users
$du -a > ~/Desktop/new_files.txt
$opendiff ~/Desktop/old_files.txt ~/Desktop/new_files.txt

The last command takes some time to load when both the text files are around 14mb. The result is a side-by-side view of both file lists, with anything that differs in size or existence highlighted. I was able to confirm that all the files I care about are present, and the only things missing are cache files, with a few exceptions.

The first thing I noticed is that the iTunes Music Library.xml file was missing on the new drive. Apparently this is normal and iTunes will recreate it from the iTunes Library.itl file.

The second thing I noticed was that several word documents differed in size. I'm not sure why, but if I open them I don't see any corruption, and after closing the file on the new drive, its size becomes the same as the original (without even saving).

The third problem was IMG_6589, which was not restored by the backup. This is a random picture out of the many thousands I have. I tried to open it on the old drive, and it opens but appears corrupted in preview. If I try to copy it, I get an error about "some data can't be read or written" and an error code -36. I guess this file got corrupted on the old drive so Time Machine skipped it because it wouldn't copy. It's concerning that files can just rot away and then be deleted from the backup! Perhaps this is a bug in Time Machine.