Need to recover a zillion erased files

[Bcowart]

I need some hard drive recovery help. Here's the situation:

I was using a program called PhoneView in order to copy out some data from my iPhone, and was simply copying Notes files over to a new folder on an external Seagate 1.5TB drive I got at Costco. Sort of like a WD Passport. Very quiet, etc. The drive has been working fine for about six months, attached to my new iMac. The drive has all my data on it.

Anyway, as I was doing a simple drag-and-drop operation from the PhoneView app, I noticed that folders were disappearing right in front of my incredulous eyes. Finder was open, and all the folders on the Seagate were evaporating as though I had selected all, and then deleted. I didn't want to stop the process by ejecting the disk for fear of messing up the directory, so I let it finish.

After all the folders were gone, what remained were the Notes files I had wanted to copy over from the iPhone. I happen to know that over 80,000 files were on the drive, including all my music, photos, videos, data files of various types such as PaperPort, and my virtual machines, etc. So, this is a major bummer. I have backups but not very recent. I don't want to have to reconstruct the whole disk from other drives. I am thinking that unerring the files should be trivial if I can just reverse marking of the files as deleted in the file allocation table. The drive is formatted as Mac OS Extended (journaled). I am running Lion 10.7.4.

So far, I have had no luck. I looked in the trash first, but nothing was there. I have tried Disk Warrior which basically found nothing significantly wrong with the FAT, save a few odds and ends. The proposed repair looked virtually like the current one. That is to say, only the Notes files were there, none of my 80,000+ missing files were. So, next up, I tried File Savage and Remo Recover. Neither of them was helpful, really. File Salvage did find ALL the files, amazingly. It took all day but it reported 80K+ files. Unfortunately, they didn't have names that made sense (just numbers), and from what I could tell, folder nesting info was lost. Considering that a lot of these files are iTunes music files, and photos, losing the folder names and locations would be a disaster. Remo seems to work the same way. Basically, these apps are designed to recover individual files and not reverse an unintentional volume wipe. I had great hopes for Disk Warrior, but was surprised that it didn't find an old FAT from prior to the erasure. Sigh.

I have written nothing to the drive at this point, as far as I know. It IS mounted, though I am being careful to unmount it when not trying to roll it back.

One possibly related point: I have a utility called "Paragon NTFS driver for Mac OS X", though I was not using it with this drive. I have some other NTFS-formatted WD MyBook drives I use that with. Those drives were not turned on when this happened. I can't imagine this has anything to do with the erasure.

Does anyone here have a suggestion for me? I'm pretty desperate.

Thanks!
Bob Cowart
(Author of 47 books about computing, mostly Windows books, and a relatively recent Switcher)

 

[Tumbleweed666]

Did you try Data Rescue 3 ? I'd say its pretty much recognised as 'the best'.

BTW, you wrote 47 books on computers and not one of those covered backup ?

 

[Consultant]

Open "Disk Utility" app.

Click on the drive on the left pane

Click "Verify Disk" button, and repair disk if necessary.

 

[Bcowart]

Did you try Data Rescue 3 ? I'd say its pretty much recognised as 'the best'.

BTW, you wrote 47 books on computers and not one of those covered backup ?

Thanks. I'll check it out.

Yes. Good point. Yeah, caught with my pants down! I covered backup many times in the books, of course. I only recently got all my data onto this drive and was still figuring out how to do the usual mirroring I do on my Windows boxes. I usually have two identical drives and a full-time mirroring setup. Due to some limitations of what kind of drives Time Machine can use, I was still researching which way to do real time backup of a Mac. Still not sure. Also, I was testing out the Seagate to see how reliable it was going to be. Seems better than the WD's I have, assuming this recent snafu was not a hardware problem.

 

[Fishrrman]

"File Salvage did find ALL the files, amazingly. It took all day but it reported 80K+ files. Unfortunately, they didn't have names that made sense (just numbers), and from what I could tell, folder nesting info was lost."

The answers to your problem are in the excerpt above.

The "File Salvage" app did its job -- it found ALL your files, still there.

BUT -- you also noticed that filename and folder hierarchies have been lost. This is normal for "file recovery". Let me shout that out: THIS IS NORMAL. You will do no better than this.

Why is that, you're wondering?

Think for a moment what happens when a drive is "erased". What really happens?

The -directory- is erased, "wiped clean". (Not the data out on the sectors of the drive, which remains intact unless you also choose to "zero out" the drive, which you did not do).

Where are file "names" usually stored? Where are folder hierarchies constructed?
In the -directory-, of course.

When you erased the drive, you destroyed this information (even though the data was left alone).

What the data recovery software did, is to "look beyond" the directory (because there is no useful directory) and go "right to the platters" of the drive. The recovery software scavenges the sectors of the drive (which is why it takes so long), and then slowly sorts and re-constructs the scraps of info it finds into "whole files" again, as best it can.

However, the file recovery software has "no knowledge" of the previous names or folder hierarchies, because they were in the directory, which is gone. So instead, the recovery software assigns numbers or "temp names" for the files it reconstructs.

So -- if "File Salvage" found the files, even without proper names or folders, it has done all it can do for you. And apparently, it did its job and did it well.

Yes, renaming the files and re-orienting them into proper folders will be a LOT of work for you. But consider -- at least you got the files back, right? Many folks can't get this far.

Having written that, here's a suggestion for the iTunes files:

I had a drive partition "go south" on me, but was able to recover most of the music files that were on it. Many were un-named, could not tell what they were.

BUT -- when a file is added to iTunes, iTunes seems to "sort it" by using the METADATA contained in the file, not the file name per se.

And chances are the metadata "came through" the file recovery ok -- even though you can't see this by looking at the (now changed) file names.

The solution for me was to dump the entire mass of mp3 files into a new iTunes libarary, and then give iTunes the time to "sort it all out" (again, by using the metadata). Amazingly, iTunes reconstructed albums, song names, etc. Not 100%, but it did a surprisingly good job.

This might work for you....

 

[Bcowart]

Thanks for taking the time to respond.

My understanding of a journaled DOS was that it kept info about file and folder arrangements as well as any file alterations (edited files, for example) so that a 'rollback' could be done. For example, if the FAT gets incompletely written due to a power outage.

The help file for Disk Warrior sort of implies as much. Sort of. I was hoping that more than one copy of the FAT was stored on disk, and what needed to be done was a rollback. Disk Warrior looks for errors in the FAT and corrects them, creates a new FAT temporarily, and upon your permission replaces the broken one with the new one. I tried DW and it didn't have much to offer, as noted in my first post.

I understand what you are saying about the folder hierarchy info being lost because that is what got erased, not the target files. I just thought there might be a backup FAT hanging around, or that the deleted folders and files just had a flag on them that got set as "deleted" and that flipping the flags back was relatively easy, so long as nothing new had been written to disk, meaning the data files were still intact, no missing sectors.

I don't recall whether I did this or not, but Ctrl-Z or Edit/Undo delete might have worked, if the cache for recording what was just done (delete all 80K+ files) had been big enough to store all that info, which I kind of doubt.

I think I was hopeful because of some features in NTFS, a Windows-based journaling DOS. WIkipedia shows this:

http://en.wikipedia.org/wiki/Transactional_NTFS

This article applies to apps being able to abort an entire transaction if even a small error or incompleteness in a transaction is noticed by the OS or app. It just says "nuh-uh" and rolls back to the FAT as it was before the modifications were begun. Email and some other Internet transactions (e.g. FTP) have had this ability for ages, preventing mangled or incomplete data exchange between server and client. It's a powerful feature in an OS.

However, that's not applicable to my case, where everything was 'erased', then the FAT was successfully rewritten. What should have been possible, but seemed to not be possible, was a BIG 'undo' option from the Finder. OR, some of those files should have ended up in the Trash. For some reason, NONE of the files showed up in the trash. I wasn't asked the usual "xxx is too large for the Trash, do you want to just kill it?" (approximately)

Looks like I'm probably hosed. I may try sorting the found files by date and seeing what was new since I put that disk on the iMac. THen I'll use the previous backup from a few months ago when all my data was on Windows external drive(s). It might not be too onerous.

By the way, there is an add-on to iTunes called TuneUp that compares unidentified MP3 files with data on the 'Net and then modifies the tags in your MP3's to match. My sense of it is not that it looks for hidden tags in your nameless files, but rather it looks for patterns in the digital file, length of the MP3, etc, and compares it so a big database in the sky. If a match is found and you approve, it will fill in the tags and reorganize as necessary to make sense of all those number-only songs.

 

[Tumbleweed666]

Thanks. I'll check it out.

Yes. Good point. Yeah, caught with my pants down! I covered backup many times in the books, of course. I only recently got all my data onto this drive and was still figuring out how to do the usual mirroring I do on my Windows boxes. I usually have two identical drives and a full-time mirroring setup. Due to some limitations of what kind of drives Time Machine can use, I was still researching which way to do real time backup of a Mac. Still not sure. Also, I was testing out the Seagate to see how reliable it was going to be. Seems better than the WD's I have, assuming this recent snafu was not a hardware problem.

Mirroring is NOT backup !!!!

In the situation your encountered for example, where perhaps a glitch on behalf of the app you were using, or a bug, or finger trouble deleted your files, both drives would have had identical data since that's what mirroring does, so you'd still be in the exact same mess, you'd just have two copies of it


The easiest, cheapest and simplest way to do real time backup (where real time = 1 hour) is Time Machine, or you could use CC Cloner or SuperDuper with parameters set to do hourly incrementals, they can have advantages (or you can use both). Time Machine can use pretty much any directly connected drive, so I dont know what you mean by restrictions on the type of drives.

You should also have off site backup, since a TM or other backup right next to your computer isn't much use in case of fire, flood or theft. There are numerous options to make backups into "the cloud". I use Jungle Disk backing up onto Amazon, there is also Drop Box, Mozy and many others.

 

[Bcowart]

Mirroring is NOT backup !!!!

In the situation your encountered for example, where perhaps a glitch on behalf of the app you were using, or a bug, or finger trouble deleted your files, both drives would have had identical data since that's what mirroring does, so you'd still be in the exact same mess, you'd just have two copies of it


The easiest, cheapest and simplest way to do real time backup (where real time = 1 hour) is Time Machine, or you could use CC Cloner or SuperDuper with parameters set to do hourly incrementals, they can have advantages (or you can use both). Time Machine can use pretty much any directly connected drive, so I dont know what you mean by restrictions on the type of drives.

You should also have off site backup, since a TM or other backup right next to your computer isn't much use in case of fire, flood or theft. There are numerous options to make backups into "the cloud". I use Jungle Disk backing up onto Amazon, there is also Drop Box, Mozy and many others.

True enough. That was the problem with using a mirror. However, the lame program I was using on my dual 1TB drives under Win7, which came with the drives from WD was not very speedy. I often turned off the mirroring during a work day, and then as long as nothing horrific happened during the day, I'd turn it on and let it update the backup drive. The WD program was called Memeo Backup, I believe. I use SkyDrive, Dropbox, and iCloud to keep some prime files backed up in a different site. Being new to OS X, I'm just beginning to check out Time Machine.

The disk limitation I was referring to is that the volume has to be formatted HPFS. I often have NTFS drives so I can share them between OS X and Windows while running VMWare or Parallels. I use Paragon's NTFS for Mac. So that was annoying to realize. Even with the Paragon driver allowing read and write on the external drive, TM didn't want to use the drive.

I think I better just devote a large (HSF+) drive to TM and use VMware and then let TM backup as it does, including my VM file. In the meantime, i will reconstruct the missing data from my old(ish) backup drives. Funzies.

THanks for your help.

Bob

 

[throAU]

I do time machine backups and for VMs i back up seperately (excluded from TM) using the apple "backup" tool as required.


As you've discovered though, even a non-optimal solution for backing up is a lot better than nothing at all. And human error is often far more likely to bite you than hardware failure.

 

[Bcowart]

I do time machine backups and for VMs i back up seperately (excluded from TM) using the apple "backup" tool as required.


As you've discovered though, even a non-optimal solution for backing up is a lot better than nothing at all. And human error is often far more likely to bite you than hardware failure.

I remember backing up CP/M systems, and early DOS boxes. I was using tape at that point, because hard drives were so expensive. I think my first hard drive was 5 meg and it cost $600 or something crazy like that. What i do nowadays is purchase two drives of identical size, so I can just duplicate my data drive some way or another.

There was a Windows-based program that works like TM does. Unfortunately, it created a second partition on the HD and then did incremental backups where you could roll back docs as needed, from any point in time. Later, when i decided to axe that program, it wouldn't remove the partition it had created. Very annoying 'feature' of that backup program. Others had that prob, too. In general, I find backup a pain, as do many.