We're starting out the new year at our small design studio re-analyzing our need for archiving completed projects to tape.
While relatively inexpensive regarding the cost/GB - we do have to nurse the aging MacPro. Retrospect is finicky - it will frequently lose the connection to the Drobo (where the completed projects are stored) or it will lose the connection to the SCSI bus. At best, this requires a restart, at worst, it interrupts the archiving process creating a multitude of verification errors, etc.
So, while we certainly have a functional system in place, I've been tasked with at least researching our alternatives. We already us CrashPlanPro to backup each Mac (7 in total) to the cloud. Our Drobo also gets backed up as NAS connected to the machine running Retrospect. So, in a sense, this stuff is already in the cloud.
Here's our dream scenario - a project is finished, collected, pruned of unnecessary files and moved to a folder on the artist's workstation called "Completed" - this folder is "watched" for changes. When new files are added, they get uploaded to this cloud service. Here's the kicker. After they're uploaded, we want to delete them from the local "Completed" folder and have the files stored long-term with the cloud service. Ideally, retrieval would be no more difficult than tape - search for a job number, file name, etc., and download only what is necessary. Retrieval could be a rare occurrence, so it is feasible that these items move to "cold" storage.
I can sort of get there with CrashPlan - but it requires that each artist move their files to the Drobo - which currently they already do. But as the Drobo fills up - we remove older files. I believe CrashPlan will store deleted files for up to a year. But at some point, projects removed from the Drobo (or other source device) would be removed from the cloud.
I've looked into Amazon Glacier as an option - but that is about as far as I've gotten. The process seems horribly convoluted - and to even upload/retrieve in any meaningful and user-friendly manner will require the use of software such as Arq or Syncovery. Would much rather this be a set-it-and-forget-it process.
We typically have project folders that go into the 2-5GB range. Our Drobo currently houses about 3TB of completed projects, mostly for easy access. Our most recent batch of tapes which I started in June 2013 is up to 8TB. So we'll obviously need a somewhat hefty amount of space to store this stuff long-term. But we're not talking enterprise-grade. Most of the services that I've found for long-term storage like this are enterprise-level for cold storage of server assets and the like. Nothing for creatives who work with large batches of large files.
Anyhow - just thought I'd throw this out there for some discussion. I may have missed a service, or other obvious solution that others have tried or are implementing. I welcome all thoughts and feedback. Thanks!
While relatively inexpensive regarding the cost/GB - we do have to nurse the aging MacPro. Retrospect is finicky - it will frequently lose the connection to the Drobo (where the completed projects are stored) or it will lose the connection to the SCSI bus. At best, this requires a restart, at worst, it interrupts the archiving process creating a multitude of verification errors, etc.
So, while we certainly have a functional system in place, I've been tasked with at least researching our alternatives. We already us CrashPlanPro to backup each Mac (7 in total) to the cloud. Our Drobo also gets backed up as NAS connected to the machine running Retrospect. So, in a sense, this stuff is already in the cloud.
Here's our dream scenario - a project is finished, collected, pruned of unnecessary files and moved to a folder on the artist's workstation called "Completed" - this folder is "watched" for changes. When new files are added, they get uploaded to this cloud service. Here's the kicker. After they're uploaded, we want to delete them from the local "Completed" folder and have the files stored long-term with the cloud service. Ideally, retrieval would be no more difficult than tape - search for a job number, file name, etc., and download only what is necessary. Retrieval could be a rare occurrence, so it is feasible that these items move to "cold" storage.
I can sort of get there with CrashPlan - but it requires that each artist move their files to the Drobo - which currently they already do. But as the Drobo fills up - we remove older files. I believe CrashPlan will store deleted files for up to a year. But at some point, projects removed from the Drobo (or other source device) would be removed from the cloud.
I've looked into Amazon Glacier as an option - but that is about as far as I've gotten. The process seems horribly convoluted - and to even upload/retrieve in any meaningful and user-friendly manner will require the use of software such as Arq or Syncovery. Would much rather this be a set-it-and-forget-it process.
We typically have project folders that go into the 2-5GB range. Our Drobo currently houses about 3TB of completed projects, mostly for easy access. Our most recent batch of tapes which I started in June 2013 is up to 8TB. So we'll obviously need a somewhat hefty amount of space to store this stuff long-term. But we're not talking enterprise-grade. Most of the services that I've found for long-term storage like this are enterprise-level for cold storage of server assets and the like. Nothing for creatives who work with large batches of large files.
Anyhow - just thought I'd throw this out there for some discussion. I may have missed a service, or other obvious solution that others have tried or are implementing. I welcome all thoughts and feedback. Thanks!