Aperture trouble after hard drive crash (with Time Machine backup)

[rusty2192]

Hi guys,

The hard drive in my iMac just crashed last weekend. No big deal, thats what Time Machine and AppleCare are for. So after a trip to the genius bar, I restored from my Time Machine backup. Everything seemed to be going great until I tried opening Aperture for the first time. That was when I got the error: "There was an error opening the database for the library." I let Aperture manage the files, not reference.

The entire library is there, including all masters and even versions (with previews). It appears to be a problem with just the database. Using MRoogle. I found this thread from a while back. I have tried all of the suggestions but rebuilding the database (more on that later) and nothing has worked yet. When I remove the plist file, Aperture opens up fine and starts a new library. I then tried to switch to the old one. It pulls up a progress bar saying "Upgrading library: Scanning library" but never does anything. I let it sit like this all day while I was at work and when I got home, it was still just sitting there.

OK, now back to rebuilding the database. It let me do the command-control-click thing once or twice, but now I cannot get it to open that pane when it is set to open the old library. Oddly enough, when I remove the plist so that it tries starting a new library I can get to it every time.

One other note, just in case it may have something to do with it: When the hdd died, i was on Aperture 3.1.2. While it was down, 3.1.3 came out. when I restored it, the first thing I did was run Software Update, which then updated Aperture. I never tried opening the library in between.

OK, so finally, does anyone have any idea what I can do to get my library back? If it's beyond all hope, so be it. All of my masters are there and I can always just import them into a new library.

One last question: what should I be doing to prevent such a problem in the future?

Thanks in advance for any help.

 

[Ruahrc]

One last question: what should I be doing to prevent such a problem in the future?

Personally the first thing I'd do is can Aperture's managed library and go with the referenced style where you will always be able to see your master files in folder structures you define. Maybe it's an archaic way of thinking but the number of threads here and elsewhere outlining problems with their crashed/corrupted/unrecoverable/messed up Aperture managed libraries makes me glad I prefer to organize my files myself.

Next I'd come up with some way to independently back up the Aperture library, aside from Time Machine. I don't know how it is now, but I remember in the early releases of Aperture, the library did not play well with Aperture and Apple even recommended for users to exclude the Aperture library from the TM backup and do it manually. The reason is/was that the Aperture library was one giant file which got updated very often- nearly every time you made a change within the program. TM only backs up on a file-by-file basis, so every time a small change was made in the aperture library (which was basically always) TM would re-back up the whole thing over and over, chewing through gigs and gigs of HD space and also making the backups take a long time. I don't know what the eventual resolution to this problem was (or if there ever was one) but TM's backup method has not changed since it's introduction (still file-by-file basis) and I don't think Aperture's library format has changed either.

Another issue was that since it took so long to back it up, there were problems related to TM trying to back up the library file while it was in use (if Aperture was running while TM was going in the background) and the resulting problem of what was essentially a continually-changing file being unable to be reliably backed up with TM.

There is a similar situation with LR (which is what I use) where the catalog file is one large file on the HD, and is updated very often when the program is in use. I exclude the LR library folder from TM, and LR has its own built-in catalog backup feature which will back up the catalog to external HD or any other location at an interval you define.

Ruahrc

 

[flosseR]

Personally the first thing I'd do is can Aperture's managed library and go with the referenced style where you will always be able to see your master files in folder structures you define...

Ruahrc

+1 on Ruahrc's point. I wrote extensively on how to do this and even automate any future imports to do it. I also use iBackup (free) to back up the referenced files to 2 external drives.. I find Time Machine ok but not great, especially not for this kind of thing. Consider using Aperture Vaults instead with a referenced system..

 

[initialsBB]

So in essence, time machine is not a valid backup solution for aperture libraries ?

Do the aperture vaults on external hard drives help, or will they also have a database problem ? I have 2 external vaults, using a managed library, and a time machine backup. Should I change my backup strategy ? Can I exclude Aperture from my time machine backup, seeing as it's obviously broken ?

 

[rusty2192]

+1 on Ruahrc's point. I wrote extensively on how to do this and even automate any future imports to do it. I also use iBackup (free) to back up the referenced files to 2 external drives.. I find Time Machine ok but not great, especially not for this kind of thing. Consider using Aperture Vaults instead with a referenced system..

Looks like I'll be switching to referenced files and using the vault. Great write up by the way!

I am still trying to revive the old library. Anyone have any clue?

 

[rusty2192]

UPDATE: I got it back. I ended up totally deleting Aperture following Apple's instructions here and then reinstalling 3.0 from my disk. When I tried to open the library it said it was unable to since it was made by a newer version of Aperture (3.1.2), so I found the 3.1.2 update dmg from Apple and installed that. The library still wouldn't open, but this time it did allow me to hold Command and Option while opening the app. A quick rebuild of the library and it roared to life.

Now to create a vault and then figure out how I want to administer reference files from now on.

Thanks for all of the help guys.

 

[initialsBB]

Now to create a vault and then figure out how I want to administer reference files from now on.

I've been looking in the package contents of an aperture vault, and it is a cluster f*£% to extract anything meaningful if Aperture goes balls up and you have to access the masters by some other means.

I'm not so sure that aperture vaults are very good backups after all...

 

[Clark Kent]

I've been looking in the package contents of an aperture vault, and it is a cluster f*£% to extract anything meaningful if Aperture goes balls up and you have to access the masters by some other means.

I'm not so sure that aperture vaults are very good backups after all...

So vaults aren't good because the Aperture package isn't easy to dig through and find what you want? Vaults are awesome because you can restore your entire library to the point in time you last updated.

If you have a good backup strategy (multiple vaults plus at least one clone or time machine backup) then getting your master files out of Aperture is as easy as pie. Just export them after getting everything restored.

If a person cannot reliably backup Aperture, they are probably someone that shouldn't be using it in the first place.

 

[initialsBB]

Wirelessly posted (iPhone: Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5)

Vaults are fine as long as Aperture is working. Imagine Apple cans Aperture tomorrow, that Vault suddenly becomes completely useless. I'll keep using Vaults as one of the backup solutions, but I'm definitely looking into diversifying and opening up my library storage methods.

I'll probably exclude Aperture from Time Machine, switch to a referenced library, create a CCC/SuperDuper script to sync the folder somewhere, on top of the 2 vaults... If this is at all possible.

 

[flosseR]

Personally, I use vaults to back up my Library, not my RAW files. I have my referenced RAW files under MY control and the Library which contains all the edits and previews etc. in Vaults. You CAN btw restore a vault completely without any library and that's why I think they are great because if you have a vault with 10000 images its a few GB, which you restore fast. If you have moved your referenced files you can re-locate them and be done in less than an hour of recovery.. now if your RAWs are in your library and that gets screwed up.. you will take a lot more than an hour..

 

[Clark Kent]

Wirelessly posted (iPhone: Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_3_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5)

Vaults are fine as long as Aperture is working. Imagine Apple cans Aperture tomorrow, that Vault suddenly becomes completely useless. I'll keep using Vaults as one of the backup solutions, but I'm definitely looking into diversifying and opening up my library storage methods.

I'll probably exclude Aperture from Time Machine, switch to a referenced library, create a CCC/SuperDuper script to sync the folder somewhere, on top of the 2 vaults... If this is at all possible.

How exactly would it become completely useless? If Apple suddenly dropped Aperture from their lineup, the program would still work on your computer and you would still have all access to your files. Your vault would still be a reliable backup. You would be free to continue using the program in its final version or pull your files out and move them to another program.

 

[Ruahrc]

Doesn't having multiple vaults of managed libraries mean you are (redundantly) storing multiple copies of your master photos?

Getting your masters out of a referenced library is not even necessary since they're already out. Ready to move to a new computer, drive, or software suite if you want. Getting masters out of a managed library requires you to load (or even install/restore) Aperture and then export all your masters back out. Extra unnecessary steps.

The referenced library favors the easiest-access, most platform-independent and decentralized approach to file archival.

Ruahrc

 

[Clark Kent]

Doesn't having multiple vaults of managed libraries mean you are (redundantly) storing multiple copies of your master photos?

So you only have 1 backup copy of your photos?

I have three vaults.

One is kept offsite and updated every 1-2 weeks.

Another is on a backup drive at home. I have a partition with time machine as well but I keep a vault because I have plenty of space and it gives me flexibility.

The final vault is on a partitioned drive I keep connected to my PS3 and it's for quick backups after any imports or edits. It's in the same room as my computer so it only takes minutes to grab it and update.

If my computer dies, I'm safe. If someone breaks in and steals my electronics, I'm safe. If my house burns down, I'm safe.

 

[initialsBB]

I can't seem to find a way to move from a managed to library to a referenced one. The Consolidate command only proposes to move/copy into the library, but not out of the library ?

 

[Clark Kent]

I can't seem to find a way to move from a managed to library to a referenced one. The Consolidate command only proposes to move/copy into the library, but not out of the library ?

Try the Relocate Masters command directly above the Consolidate Masters one.

 

[initialsBB]

Try the Relocate Masters command directly above the Consolidate Masters one.

Ah OK, thanks, I didn't understand "relocate" that way.

What do you reckon is the best setting for subfolder format ? Straight image year/month/date then master file name ?

 

[steve-p]

Next I'd come up with some way to independently back up the Aperture library, aside from Time Machine. I don't know how it is now, but I remember in the early releases of Aperture, the library did not play well with Aperture and Apple even recommended for users to exclude the Aperture library from the TM backup and do it manually. The reason is/was that the Aperture library was one giant file which got updated very often- nearly every time you made a change within the program. TM only backs up on a file-by-file basis, so every time a small change was made in the aperture library (which was basically always) TM would re-back up the whole thing over and over, chewing through gigs and gigs of HD space and also making the backups take a long time. I don't know what the eventual resolution to this problem was (or if there ever was one) but TM's backup method has not changed since it's introduction (still file-by-file basis) and I don't think Aperture's library format has changed either.

I may be wrong but I think that information is out of date. My understanding of it at the time was that actually the library is not one big file at all. It appears that way in the finder but it's really a filesystem in it's own right. The issue with Time Machine was that it was not seeing it this way and therefore backing up the library as if it were one big file, rather than looking inside and doing proper incremental backups. AFAIK this is no longer an issue.

Having said that, I agree totally that referenced masters is the way to go, despite it seeming tempting to let Aperture do it. The main reason I use referenced masters is that in the absolute worst case scenario, the masters should be usable on any backup, even if all the adjustments are lost. I keep my masters in a structured way inside my Dropbox folder so not only are they backed up in the cloud, they are also present on my other computers even though I don't use them there. I have about 90GB of masters at the moment. I don't use Aperture vaults, just referenced masters and allow TM to back up the library which seems to work.

I also had a hard disk failure a couple of months ago and restored from a TM backup. I still had some issues. A lot of the previews and thumbnails were corrupt after the restore, but I still had the referenced masters and the adjustments in the database were all intact, so I just had to force Aperture to reprocess everything and that sorted it.

 

[initialsBB]

My understanding of it at the time was that actually the library is not one big file at all. It appears that way in the finder but it's really a filesystem in it's own right.

It's an OS X Package. If you do a right-click on the file, go to "Show Package contents" you can browse through it.

I have about 90GB of masters at the moment. I don't use Aperture vaults, just referenced masters and allow TM to back up the library which seems to work.

I also had a hard disk failure a couple of months ago and restored from a TM backup. I still had some issues. A lot of the previews and thumbnails were corrupt after the restore, but I still had the referenced masters and the adjustments in the database were all intact, so I just had to force Aperture to reprocess everything and that sorted it.

I use Aperture vaults as I've seen it compresses my library (I go from 15 GB down to 6 GB in the vault) and this makes it easier on my upload.
I'm in the process of reorganizing my backup strategy. I now store my referenced shots, with a PROJECT NAME/YYYY/MM/DD subfolder format, on a RAID 1 and I have a vault next to it. My Library stays in my user's photo folder and gets backed up via Time Machine.

Both the referenced masters and the vault on the RAID 1 are synced to a spare HD, and being backed up to Amazon S3 via an app I'm testing called Arq http://www.haystacksoftware.com/arq/. Amazon is much more flexible than Dropbox in it's pricing, and with a 30 day demo for the Arq client, I'm not tied for 12 or 24 months. Apparently their mac compatibility is flawless and I double checked that the backups are indeed encrypted.

The 65 GB are taking a while to upload even though I have very descent bandwidth... hope it will all run smoothly when it can just sync the content.

 

[steve-p]

I will need a new solution soon, as I will outgrow Dropbox capacity this summer, but I really need multiplatform backup for other uses than pictures. I also like having everything automatically synced to multiple machines, both here and off-site, and Dropbox is really good at that stuff. For example it automatically syncs a copy to a private server in the office which is in a secure hosting centre, so if Dropbox goes bust or is down for a while, at least I have multiple copies of everything in more than one location already. However, Arq might still be attractive for part of what I back up. I was kind of hoping that Apple were going to be providing something similar as part of iCloud though.

 

[initialsBB]

I was kind of hoping that Apple were going to be providing something similar as part of iCloud though.

I'm still waiting to see what exactly they announce. I'm hoping it's good. In the meantime Arq let' me backup right now in a pay as you go style contract. It's the perfect stop gap as I hold out to see what Apple bring to the table.