Register FAQ / Rules Forum Spy Search Today's Posts Mark Forums Read
Go Back   MacRumors Forums > Special Interests > Visual Media > Digital Photography

Reply
 
Thread Tools Search this Thread Display Modes
Old Jul 12, 2011, 10:43 AM   #1
mousemd
macrumors member
 
Join Date: Nov 2002
Aperture crash, lost master files

I had a HORRIBLE crash with my Mac resulting in the rebuilding of my iMac OS directory with Diskwarrior (thank you man!). Unfortunately, it resulted in the inability to open aperture (due to plug-in failure from what I could gather from the internet/forums). Nonetheless, I reinstalled aperture 3 and was able to recover my aperture library. Now, here is the tough part:

When I opened Aperture now, there are many BLANKS where the pictures used to exist before. It appears that the master files are gone. I believe I kept my library as a MANAGED (default) not referenced but I could be wrong. Nonetheless, I can NOT find the pics that are missing in the library. I managed to find SOME of the pics on my hard drive under a "RECOVERED" folder, but not all of my pics are there. I tried REBUILDING the library, but it did not help with those files that were missing master pics. In fact, instead of showing the # of pics that should be in those projects, it now just shows a BIG FAT 0 for what used to be there.

I THOUGHT I had a backup using Time Machine, but apparently TIME MACHINE never made any back ups (ie the Library (sans some of the files weighs in at 60gig+, the Time Machine backups are like 5 gig).

So, am I totally screwed now?

Thanks for your input guys. I put a question up on the Apple Forums, but didn't get much of a response.
mousemd is offline   0 Reply With Quote
Old Jul 12, 2011, 01:15 PM   #2
compuwar
macrumors 601
 
compuwar's Avatar
 
Join Date: Oct 2006
Location: Northern/Central VA
You are likely screwed, which is why I always use REFERENCED files- disk recovery software can recover referenced files, for managed files, it needs to recover the database, and that's less likely- I'd rather have as many of my files as possible recoverable. I also back up my files to a separate disk when I upload them to the computer. That disk gets backed up to an external that goes off-site. I'm actually considering cloud storage as an additional layer of backup.

Why am I telling you this? Because as you have found out the hard way, it's up to you to ensure your files are backed up. Too many backups is way better than too few. Also, you need to TEST your backups at some point when you still have all the originals, not after you've lost them. Hopefully others can learn from your mistakes, and hopefully going forward you'll know what to do.

Paul
__________________
PaulDRobertson.net Have I been helpful? How about a print for your wall? US, UK and CA fine art sales!
compuwar is offline   1 Reply With Quote
Old Jul 12, 2011, 03:01 PM   #3
mousemd
Thread Starter
macrumors member
 
Join Date: Nov 2002
Thanks for the response. I have been backing up religiously since a previous problem 6 years ago where I lost some photos as well. I guess it never occurred to me to VERIFY that the file SIZE was correct on the time machine backup. I always assumed that if the folder was there, the file would be correct as well.

In either case, I am on my way of switching to Lightroom and making a cloned drive (thanks Superduper). If by the off chance I don't switch, how can I change my MANAGED files to a REFERENCED File?

Funny thing, on my way to transferring my files OUT of Aperture, someone on the web suggested that I switch out RAW, RAW adjusted, JPG, TIFF, and other files. This method is nice, but the output files are approaching a TERABYTE of data (ok several hundred GIGABYTES). Is this right?!?!?! My Aperture library is less 100GB.

Quote:
Originally Posted by compuwar View Post
You are likely screwed, which is why I always use REFERENCED files- disk recovery software can recover referenced files, for managed files, it needs to recover the database, and that's less likely- I'd rather have as many of my files as possible recoverable. I also back up my files to a separate disk when I upload them to the computer. That disk gets backed up to an external that goes off-site. I'm actually considering cloud storage as an additional layer of backup.

Why am I telling you this? Because as you have found out the hard way, it's up to you to ensure your files are backed up. Too many backups is way better than too few. Also, you need to TEST your backups at some point when you still have all the originals, not after you've lost them. Hopefully others can learn from your mistakes, and hopefully going forward you'll know what to do.

Paul
mousemd is offline   0 Reply With Quote
Old Jul 12, 2011, 03:30 PM   #4
compuwar
macrumors 601
 
compuwar's Avatar
 
Join Date: Oct 2006
Location: Northern/Central VA
Quote:
Originally Posted by mousemd View Post
Thanks for the response. I have been backing up religiously since a previous problem 6 years ago where I lost some photos as well. I guess it never occurred to me to VERIFY that the file SIZE was correct on the time machine backup. I always assumed that if the folder was there, the file would be correct as well.
I've seen enough failed backups that I generally manually back up, if I didn't, I'd want to test restore.

Quote:
Originally Posted by mousemd View Post
In either case, I am on my way of switching to Lightroom and making a cloned drive (thanks Superduper). If by the off chance I don't switch, how can I change my MANAGED files to a REFERENCED File?
http://aperture.maccreate.com/2010/0...d-and-managed/

Quote:
Originally Posted by mousemd View Post
Funny thing, on my way to transferring my files OUT of Aperture, someone on the web suggested that I switch out RAW, RAW adjusted, JPG, TIFF, and other files. This method is nice, but the output files are approaching a TERABYTE of data (ok several hundred GIGABYTES). Is this right?!?!?! My Aperture library is less 100GB.
Exporting adjusted files will result in complete files being written where just changes are stored in the database. Hard drives are cheap though- I use a USB2/SATA dual drive dock, and buy regular bare internal drives- drive capacities keep growing quickly enough that I don't run out of space as I copy drives to new ones and add images. However, I've recently gotten ready to focus stack large images, and potentially panoramas, so at that point, I'm sure I'll have to move to a second drive.

Paul
__________________
PaulDRobertson.net Have I been helpful? How about a print for your wall? US, UK and CA fine art sales!
compuwar is offline   0 Reply With Quote
Old Jul 15, 2011, 01:57 AM   #5
Clark Kent
macrumors member
 
Join Date: Apr 2009
Location: Tokyo, Japan
In the future, don't trust Time Machinecand Time Machine alone to back up irreplaceable files like photos.

Use a vault and/or a clone to be safe.
Clark Kent is offline   0 Reply With Quote
Old Jul 15, 2011, 06:10 AM   #6
mousemd
Thread Starter
macrumors member
 
Join Date: Nov 2002
Thanks Clark. I have learned my lesion for the second time now. I am backing up with vault. But in the process of switching to Ligthroom. Exporting the damn files from aperture is nuts (raw, raw a dusted, tiff, jpg) has resulted in hundreds og gigabYtes if not over a terabyte of data. Pain in the ass. So I've had to buy a bunch of hard drives in the process to compensate. My back up plans include FireWire bootable 1tb drive with super duper clone, drobo 8tb drive/time machined on it. And then another backup of the photo folder. I've been debating on using carbonate as well, but seeing how the photos are taking 1-2 days to transfer to the hard drive I can't imagine what it would be like to baack up over the Internet.

I feel like it seems a bit of overkill to have raw, raw adjusted, tiff, and jpg but I can not prove to myself that I don't need ALL of those files. Too many pics. I don't have the time to review each one.
mousemd is offline   0 Reply With Quote
Old Jul 15, 2011, 07:58 PM   #7
MacProDude
macrumors member
 
Join Date: Aug 2006
Location: SF Bay Area, CA
Your masters are probably still there. It's really just a hidden directory mechanism in OSX. Here's something you can do to find all files with a certain file extension over a certain size in your Aperture Library.

Right-click on your Aperture library and select "Show Package Contents". It'll open a finder window that shows the contents just like any other directory.

Go into the Masters subdirectory. Your master files should all be in there somewhere. Mine appears to be by year, then by month, then by import date/time. I assume that's the way it is for everyone.

If not, then you can look for large files in your Aperture library with the file name extension for your masters (*.CRW, *.CR2, *.NEF, whatever). Something like:

find . -name "*.CR2" -exec ls -l {} \; | awk '{if ($5 > 20000000) print $5" "$9}'

Replace "." with the path to your Aperture library.
Replace "CR2" with the file extension for the files you are looking for
Replace 20000000 with a number smaller than the minimum file size for your camera (20 million bytes in this case) but larger than most other files (eliminates thumbnails, etc if you're searching for .png or .jpg, etc).

It could take a while to run if your library is large. And you might need a bit more help figuring out what to do with the results, but that's a start for you, and maybe we can all help you find your originals. I certainly hope you don't have to try this second method.

Good luck.
MacProDude is offline   0 Reply With Quote
Old Jul 15, 2011, 08:08 PM   #8
MacProDude
macrumors member
 
Join Date: Aug 2006
Location: SF Bay Area, CA
Oh, sorry, I didn't read how you had to recover the files via diskwarrior. They might not be there, but that probably has nothing to do with referenced vs managed. If the filesystem is hosed, it can take out either of them because in either case, they're just simple files on the file system.
MacProDude is offline   0 Reply With Quote
Old Jul 15, 2011, 08:30 PM   #9
mousemd
Thread Starter
macrumors member
 
Join Date: Nov 2002
Macprodude

I can assure you that have done the former with opening my library and looking for the master files. The dates are there but there are 0 files under the file folders for the .cr2 files or whatever.

The latter method intrigues me because I have never thought about doing a search like that before. Is this in terminal via command line? Or some other method?
mousemd is offline   0 Reply With Quote
Old Jul 15, 2011, 08:43 PM   #10
mousemd
Thread Starter
macrumors member
 
Join Date: Nov 2002
@MacProDude -

I still don't understand how my system just died the way it did. I appreciate your input re: Diskwarrior recovery. I didn't think that it would kill the files in the folder. I thought it was just restore the directory.

Quote:
Originally Posted by MacProDude View Post
Oh, sorry, I didn't read how you had to recover the files via diskwarrior. They might not be there, but that probably has nothing to do with referenced vs managed. If the filesystem is hosed, it can take out either of them because in either case, they're just simple files on the file system.
mousemd is offline   0 Reply With Quote
Old Jul 15, 2011, 08:45 PM   #11
MacProDude
macrumors member
 
Join Date: Aug 2006
Location: SF Bay Area, CA
Yeah, it's in the terminal, aka from the shell.

find . -name "*.CR2" -exec ls -l {} \; | awk '{if ($5 > 20000000) print $5" "$9}'

Find from currently directory (.) files that fit "*.CR2". For each file found, do an "ls -l". if the fifth string on the "ls -l" result (which is the file size) is over 20,000,000 bytes, then print the fifth string (the size) a space, and then the ninth string (the path/filename).

Honestly, I forgot to remove the printing of the fifth string, as I did that just to verify the command did what I wanted. This would be fine:

find . -name "*.CR2" -exec ls -l {} \; | awk '{if ($5 > 20000000) print $9}'
MacProDude is offline   0 Reply With Quote
Old Jul 15, 2011, 08:54 PM   #12
MacProDude
macrumors member
 
Join Date: Aug 2006
Location: SF Bay Area, CA
Quote:
Originally Posted by mousemd View Post
@MacProDude -

I still don't understand how my system just died the way it did. I appreciate your input re: Diskwarrior recovery. I didn't think that it would kill the files in the folder. I thought it was just restore the directory.
I can't say I do, either.

I didn't mean to imply that diskwarrior hosed anything. It's possible that the filesystem was hosed to the point that diskwarrior simply couldn't recover everything/anything.
MacProDude is offline   0 Reply With Quote
Old Jul 15, 2011, 10:21 PM   #13
Ruahrc
macrumors 65816
 
Join Date: Jun 2009
I'm not understanding why if you had a big disk failure but had a TM backup, you didn't just reinstall the OS clean and do a Time Machine restore, which should have put everything back to a working state? Rather than try to recover what seems like a severely damaged disk using Diskwarrior and then trying to cobble things back together from there? If this didn't work, then the TM backup would have been corrupted in which case it would not be the first time I have heard such reports.

Do you still have an unadulterated TM backup (i.e. have you been backing up to TM since the restore or is your TM backup still as it is pre-crash)? If so I'd try wiping your disk, installing the OS again, and restoring from TM backup.

What kind of crash exactly was it that you suffered?

Ruahrc
Ruahrc is offline   0 Reply With Quote
Old Jul 16, 2011, 06:10 AM   #14
mousemd
Thread Starter
macrumors member
 
Join Date: Nov 2002
Yes I do have a time machine backup...sort of. Time machine FAILED to back up my Aperture library. Yes there was a copy of the file/folder, but it was INCOMPLETE. For example the damaged folder (the one I am trying to cobble together) is around 60 to 100 gb give or take. The backups on time ,aching were ~5 to 6 gb. Trust me, all I wanted to do was do a clean install and start from scratch but that would've meant even MORE lost photos.

Of course, when I rebuilt with diskwarrior, I didn't think that it would've deleted some of my photo files either. I guess the point is I didnt trust Time Machine enough just the WIPE my hard drive from th beginning and do the Time Machine restore. In retrospect, I'm glad I didn't domthatdue to the failed backups.

As far as what kind of crash I suffered, I have no idea. I was trying to boot camp into windows and suddenly I got the blue screen of death. Something about issing some files which was preventing windows to load properly. So I thought I would just boot into Mac os, but there was NO OPTION to do so at start up. I tried to use the disk utility program that came with the install disk on my iMac and the response I got with that program said that the directory was seriously damaged and that I should they and back up was many files I could and then reinstall the the OS. However I couldn't access the drive to do so. In retrospect, when I booted th iMac in target mode via FireWire using my macbook pro I should've tried and looked at the directories to see if I could recover some files, but i thought hey I have doskwarrrior and it had fixed the problem before on my other macs along with my drobo. Plus I also have a time machine backup (or so I thought). So I still have no idea how the directories became corrupted in the first place.
Quote:
Originally Posted by Ruahrc View Post
I'm not understanding why if you had a big disk failure but had a TM backup, you didn't just reinstall the OS clean and do a Time Machine restore, which should have put everything back to a working state? Rather than try to recover what seems like a severely damaged disk using Diskwarrior and then trying to cobble things back together from there? If this didn't work, then the TM backup would have been corrupted in which case it would not be the first time I have heard such reports.

Do you still have an unadulterated TM backup (i.e. have you been backing up to TM since the restore or is your TM backup still as it is pre-crash)? If so I'd try wiping your disk, installing the OS again, and restoring from TM backup.

What kind of crash exactly was it that you suffered?

Ruahrc

Last edited by mousemd; Jul 16, 2011 at 06:17 AM.
mousemd is offline   0 Reply With Quote
Old Jul 23, 2011, 10:06 AM   #15
dampsquid
macrumors newbie
 
Join Date: Jul 2011
Mousemd, I know how you feel. Yesterday I did a complete restore to revert from a Lion upgrade. Of course given a Lion upgrade was a major change, I made sure my TM backup was good and current.
Post restore the first thing I noticed was my desktop background had changed to a stock Apple one. A couple of other settings were changed too. A little odd. At some point I went to Aperture to get my preferred desktop image - and discovered it couldn't find any masters. I tried rebuilding the database etc. to no avail. I even restored an earlier Aperture library - same. Then I restored an earlier library and the actual App (it had changed recently to 3.1.3, so I made sure the app was 3.1.2 to work with a library backed up when the app was 3.1.2 - same.
Next it hit me, my library was around 14GB instead of 200 or 300GB. Right clicking on the library shows the folder structure for the masters is there - just no images, at all! And it's the same for every version Time Machine has. I have absolutely no idea why this is. Initially I blamed Lion, the irony of course is on Lion APerture and it's library was there, and working just fine.
I have already used File Juicer (on another machine) to pull the thumbnail/preview images from the library. So I have a misnamed, mis-dated version of every picture at least, but they only go to 1024 pixels.
Overall, I have lost 19,000 images covering 100 years of family history and 25 years of personal world travel. I actually feel physically sick. All of this on an Apple machine, using an Apple app, backed up with an Apple product.
As of now I'm scanning all the drives in my machine with file recovery software hoping I can at least find something to salvage. We'll see.
So, my friend, all I can offer it solidarity.
dampsquid is offline   0 Reply With Quote
Old Jul 23, 2011, 11:50 AM   #16
mousemd
Thread Starter
macrumors member
 
Join Date: Nov 2002
@dampsquid...
I feel your pain. Let me know what program u used to recover some of those photos if you do recover any. I have given up with what I was able to recover but then again I didn't try those programs you had. My only fear is that if you tried to restore and or replace your database with an older version without trying to recover first you are likely in the same boat as I am...with GONE files. I also assume u did what I did with Managed instead of a REFERENCED library. I am in the process of switching to Lightroom. It is slower in some ways and I am not used to the way it organizes the photos.I am not sure I will stick with Lightroom but after my experience with the time machine back up fiasco I am wary of the program again. I have backed now with the Vault program which is the ONLY way to back up Aperture apparenty. In any case good luck with your attempts at recovery!
mousemd is offline   0 Reply With Quote
Old Jan 21, 2013, 11:45 PM   #17
bunnkristie
macrumors newbie
 
Join Date: Jan 2013
getting my photos which is lost in a dramatic crash down of my operating system... it was cutting my nerves down.. remo photo recovery software i had used to get them back.. still i had not recovered my data back.. i only worked on the jpeg file format...
bunnkristie is offline   0 Reply With Quote
Old Jan 22, 2013, 08:11 AM   #18
Bear
macrumors G3
 
Join Date: Jul 2002
Location: Sol III - Terra
Quote:
Originally Posted by mousemd View Post
...
I THOUGHT I had a backup using Time Machine, but apparently TIME MACHINE never made any back ups (ie the Library (sans some of the files weighs in at 60gig+, the Time Machine backups are like 5 gig).
...
Just to be clear, did you check multiple backups?

Did you set Time Machine preferences to not back up any files?

----------

Quote:
Originally Posted by Clark Kent View Post
In the future, don't trust Time Machine and Time Machine alone to back up irreplaceable files like photos.

Use a vault and/or a clone to be safe.
Yes, it's sometimes good to use different backup methods.

No matter what backup methods are used, one should occasionally examine backups after they're made to make sure they look okay.
__________________
-----Bear
Bear is offline   0 Reply With Quote
Old Jan 22, 2013, 12:45 PM   #19
Laird Knox
macrumors 65816
 
Join Date: Jun 2010
I suspect the OP has moved on by now.
Laird Knox is offline   0 Reply With Quote
Old Jan 23, 2013, 04:10 PM   #20
wolfpuppies3
macrumors 6502
 
Join Date: Jun 2012
Location: Virginia, USA
Ya think? After two years?
__________________
2012 MBA 13, 2 Ghz i7, 8 GB RAM, 512 GB SSD; 2011 MBA 11, 1.8 Ghz i7, 4 GB RAM, 256 GB SSD; 2010 MB Pro 15, 2.66 Ghz i7, 8 GB RAM, 512 GB SSD; two iPad 3s and a mini, two iPhones and a bunch of iPods
wolfpuppies3 is offline   0 Reply With Quote

Reply
MacRumors Forums > Special Interests > Visual Media > Digital Photography

Tags
aperture, crash, time machine

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Similar Threads
thread Thread Starter Forum Replies Last Post
Aperture Crash mamoose124 Mac Applications and Mac App Store 3 May 7, 2014 10:41 AM
Aperture - Lost Master Files staticbool Mac Applications and Mac App Store 4 Oct 23, 2013 12:53 PM
Aperture can't find master images ReCausey Mac Applications and Mac App Store 1 Sep 3, 2012 05:10 PM
Resolved: Aperture Master File Woes Prodo123 Digital Photography 1 Jul 19, 2012 01:35 AM
Aperture: How can I delete the master after I have removed the thumbnail? falkaveli Mac Applications and Mac App Store 1 Jun 9, 2012 01:08 PM

Forum Jump

All times are GMT -5. The time now is 12:33 AM.

Mac Rumors | Mac | iPhone | iPhone Game Reviews | iPhone Apps

Mobile Version | Fixed | Fluid | Fluid HD
Copyright 2002-2013, MacRumors.com, LLC