Register FAQ / Rules Forum Spy Search Today's Posts Mark Forums Read
Go Back   MacRumors Forums > Apple Hardware > Buying Tips and Advice

Reply
 
Thread Tools Search this Thread Display Modes
Old Dec 31, 2012, 08:53 AM   #1
Dingo41
macrumors newbie
 
Join Date: Dec 2012
Is a hard drive the best way to archive video for 20-40 years

I am trying to determine the best way to archive 8mm and Hi8 video. Since these tapes degrade over time, I have been running them through an imac and putting them on 500 gig Lacie external hard drive. It seems that hard drives dont last forever, so after period of time I need to back up the external hard drives to new external hard drives. I have 6 questions:

1) Do hard drives degrade over time similar to the was that Hi8 tapes do?

2) Each time I copy one hard drive to another, is some quality of the video/data lost?

3) with what frequency should I be backing up the hard drives-- every 5 years?

4) Is an enterprise class hard drive recommended for this. My cursory research suggests that these are muh higher quality drives than for example a Lacie external hard drive.

5) which external hard drive(and manufacturer) would you recommend. I need have about 2TB of data

6) are SSD drives optimal or are these really not a proven technology yet
Dingo41 is offline   0 Reply With Quote
Old Dec 31, 2012, 09:08 AM   #2
simsaladimbamba
Guest
 
Join Date: Nov 2010
Location: located
Quote:
Originally Posted by Dingo41 View Post
1) Do hard drives degrade over time similar to the was that Hi8 tapes do?
Not in the same way, but they can fail. I have had HDDs fail on me after using them only for months, but I also have many HDDs still having an IDE interface, thus they are quite old (ten years or so) which still work.

Quote:
2) Each time I copy one hard drive to another, is some quality of the video/data lost?
Normally not, as it is a digital copy, thus a 1 stays a 1 and a 0 stays a zero.
Such copying is best done with CarbonCopyCloner, as it does a checksum (verifying everything copied okay) of the copied stuff.

Quote:
3) with what frequency should I be backing up the hard drives-- every 5 years?
There is no rule there.
I have one 500 GB HDD for my photographs (digital and analog) libraries and editing documents, one 500 GB HDD with my personal video footage in an editing friendly format.
Both 500 GB HDDs get backed up to one 1 TB HDD via CarbonCopyCloner.
And that 1 TB HDD gets backed up to another 1 TB HDD via CarbonCopyCloner.
Therefore I have three copies of my important data.
Once I register one HDD as failing, I will replace that HDD.

Quote:
4) Is an enterprise class hard drive recommended for this. My cursory research suggests that these are muh higher quality drives than for example a Lacie external hard drive.
Enterprise HDDs are made for servers or workstations and rated for 24/7 use. No need for them in your case.

Quote:
5) which external hard drive(and manufacturer) would you recommend. I need have about 2TB of data
Western Digital, Toshiba and Seagate, as they manufacture their own HDDs.

Quote:
6) are SSD drives optimal or are these really not a proven technology yet
2 TB of SSD storage would cost you at least 1000 USD nowadays, and recovering data from a failed SSD is more cumbersome than from a failed HDD.
Best to have at least two copies of the important data.
simsaladimbamba is offline   1 Reply With Quote
Old Dec 31, 2012, 09:27 AM   #3
Fishrrman
macrumors 68040
 
Join Date: Feb 2009
Have you considered M-disc technology:
http://www.milleniata.com/

Looks like an ordinary DVD, but uses a different kind of data-recording technology that is claimed to last a century...
Fishrrman is offline   1 Reply With Quote
Old Dec 31, 2012, 02:55 PM   #4
Pakaku
macrumors 6502a
 
Join Date: Aug 2009
Quote:
Originally Posted by Fishrrman View Post
Have you considered M-disc technology:
http://www.milleniata.com/

Looks like an ordinary DVD, but uses a different kind of data-recording technology that is claimed to last a century...
'Claim' is the key-word, since they have obviously never been around long enough to back up their claims



Quote:
Originally Posted by Apple fanboy View Post
No moving parts should also increase lifespan no?
You have different things to worry about, like bad sectors.
Pakaku is offline   0 Reply With Quote
Old Dec 31, 2012, 09:53 AM   #5
siraltus
macrumors newbie
 
Join Date: Nov 2009
Quote:
Originally Posted by simsaladimbamba View Post
2 TB of SSD storage would cost you at least 1000 USD nowadays, and recovering data from a failed SSD is more cumbersome than from a failed HDD.
Best to have at least two copies of the important data.
EDIT: Misread post. My bad.

Last edited by siraltus; Dec 31, 2012 at 10:20 AM.
siraltus is offline   0 Reply With Quote
Old Dec 31, 2012, 09:56 AM   #6
mobilehaathi
macrumors 601
 
mobilehaathi's Avatar
 
Join Date: Aug 2008
Location: The Anthropocene
Quote:
Originally Posted by siraltus View Post
I think you put an extra zero there by accident.

A 2TB external hard drive can be had for $120-150.

Example: http://www.newegg.com/Product/Produc...82E16822178116
No, he said SSD. He was correct.
__________________
The true way is along a rope that is not spanned high in the air, but only just above the ground. It seems intended more to cause stumbling than to be walked along.
Does size matter?
mobilehaathi is offline   1 Reply With Quote
Old Dec 31, 2012, 10:13 AM   #7
Paulywauly
macrumors 6502a
 
Paulywauly's Avatar
 
Join Date: Sep 2009
Location: Durham, UK
Hard drives aren't designed to work for decades, like all mechanical devices they will eventually fail.

You would be better off keeping backup copies on a few drives, checking them once a year and if one doesnt work just replace it with a new one.
__________________
I admit it, i spend too much on products
Paulywauly is offline   1 Reply With Quote
Old Dec 31, 2012, 10:19 AM   #8
siraltus
macrumors newbie
 
Join Date: Nov 2009
Quote:
Originally Posted by mobilehaathi View Post
No, he said SSD. He was correct.
Ah, missed the SSD part. You are right.
siraltus is offline   0 Reply With Quote
Old Dec 31, 2012, 12:44 PM   #9
gnasher729
macrumors G5
 
gnasher729's Avatar
 
Join Date: Nov 2005
Quote:
Originally Posted by Dingo41 View Post
I am trying to determine the best way to archive 8mm and Hi8 video. Since these tapes degrade over time, I have been running them through an imac and putting them on 500 gig Lacie external hard drive. It seems that hard drives dont last forever, so after period of time I need to back up the external hard drives to new external hard drives. I have 6 questions:

1) Do hard drives degrade over time similar to the was that Hi8 tapes do?

2) Each time I copy one hard drive to another, is some quality of the video/data lost?

3) with what frequency should I be backing up the hard drives-- every 5 years?

4) Is an enterprise class hard drive recommended for this. My cursory research suggests that these are muh higher quality drives than for example a Lacie external hard drive.

5) which external hard drive(and manufacturer) would you recommend. I need have about 2TB of data

6) are SSD drives optimal or are these really not a proven technology yet
0. In general: Whatever answer we give today will not be valid in ten years time.

1. If you look at ten years, who knows. For long term storage like that I would expect that some bits change for no reason whatsoever, and I would want some software that does checksumming at the very least. There's also the question how long the electronics will work. And nobody knows if sitting around and doing nothing is good or bad for a drive. I own some electronics that was high end and expensive in 1993 and that still works.

2. Each time you make a copy there might be errors introduced. There shouldn't be, but there might. That's easily checked by comparing original and copy.

3. Considering that the storage will become cheaper and cheaper over those 20 years, I'd buy a new drive once a year and keep the last few drives around.

4. Enterprise class is built to be used 24/7. You don't need that.

5. I'd take any USB 3 drive. With your long time frame, I would guess that USB 3 will be the interface that will be available for the longest time. And if I recommend company X, that company might not even be there in five years time.

6. Advantage of SSD is speed. Not really important for your purposes.
gnasher729 is offline   0 Reply With Quote
Old Dec 31, 2012, 02:39 PM   #10
Apple fanboy
macrumors 603
 
Apple fanboy's Avatar
 
Join Date: Feb 2012
Location: United Kingdom
Quote:
Originally Posted by gnasher729 View Post

6. Advantage of SSD is speed. Not really important for your purposes.
No moving parts should also increase lifespan no?
__________________
Does size matter?
Apple fanboy is offline   0 Reply With Quote
Old Dec 31, 2012, 02:58 PM   #11
rfle500
macrumors newbie
 
Join Date: Sep 2012
Quote:
Originally Posted by Apple fanboy View Post
No moving parts should also increase lifespan no?
Yes in principle, but the storage mechanism involves the confining of charge to a small physical volume. The problem is, if left for a long period of time, the charge may leak out, turning a 1into a 0, and so losing data. Both thermal effects and quantum mechanical tunnelling would contribute to leakage, and MLC flash is much more likely to degrade than enterprise SLC.
rfle500 is offline   0 Reply With Quote
Old Dec 31, 2012, 05:36 PM   #12
gnasher729
macrumors G5
 
gnasher729's Avatar
 
Join Date: Nov 2005
Quote:
Originally Posted by Apple fanboy View Post
No moving parts should also increase lifespan no?
No moving parts means no moving parts stop moving

There are other things that fail on SSD drives. And the hard drives we are talking about are not moving. Most of the time they are just sitting still, not connected to any computer. Do you know what happens if you unplug an SSD drive and leave it locked in a fire-proof safe for a year? I don't.


Quote:
Originally Posted by RedTomato View Post
Consider buying two small computers (e.g. mac mini, or PC equivalent - though mac minis seem to last almost forever; my old workplace is still happily using a 7-year old mac mini G4 as a server) that can show the video, setting them up to work properly, then just storing them with the HDDs.

Two copies can be at your house or your office. The third copy needs to be in a physically different place (in case of fire or burglary) - across town or in a different city.
Two excellent items. Your video data is of no use whatsoever if you don't have a computer that can read it and no software to process it. And to different places is important.

(Considering two different places: What would you guys think about having a Time Capsule in your neighbour's loft, and their time capsule is in your loft, only wireless access? Would require a major disaster to destroy both data and backup, and if there's a fire that burns down both your and your neighbour's home, your data backup is the least of your worries).

Last edited by gnasher729; Dec 31, 2012 at 05:42 PM.
gnasher729 is offline   0 Reply With Quote
Old Dec 31, 2012, 05:44 PM   #13
Apple fanboy
macrumors 603
 
Apple fanboy's Avatar
 
Join Date: Feb 2012
Location: United Kingdom
Quote:
Originally Posted by gnasher729 View Post
No moving parts means no moving parts stop moving

There are other things that fail on SSD drives. And the hard drives we are talking about are not moving. Most of the time they are just sitting still, not connected to any computer. Do you know what happens if you unplug an SSD drive and leave it locked in a fire-proof safe for a year? I don't.
No but I have old USB flash drives that have 5 year old stuff on them that still work fine. Isn't the technology the same? I have one which is only 256mb I found which had photos on from years back. It hadn't been used for t least 2 years and the photos were fine.
__________________
Does size matter?
Apple fanboy is offline   0 Reply With Quote
Old Dec 31, 2012, 06:24 PM   #14
SDDave2007
macrumors regular
 
Join Date: Apr 2007
Another thing not mentioned here....

What computer interface is going to be around 10 years from now to connect the "device" to?

What was state of the art 10 years AGO is mostly defunct today.... so perhaps todays media device interfaces will be obsolete 10 years from now.

Be pretty bad to have a stack of whatever type of media and nothing to read it with.
SDDave2007 is offline   0 Reply With Quote
Old Jan 2, 2013, 01:01 PM   #15
gnasher729
macrumors G5
 
gnasher729's Avatar
 
Join Date: Nov 2005
Quote:
Originally Posted by Apple fanboy View Post
No but I have old USB flash drives that have 5 year old stuff on them that still work fine. Isn't the technology the same? I have one which is only 256mb I found which had photos on from years back. It hadn't been used for t least 2 years and the photos were fine.
The individual cells have shrunk enormously in the last five years. Might be just 100 electrons nowadays that store a bit. I would trust SSD drives less for long term storage today than five years ago.

That said, the magnetic areas for bits on a hard drive have shrunk as well. Anyway, nobody actually _knows_ what happens if you store data on a brand new SSD drive and leave it for five years because nobody has tried it yet. _And_ 2 TB of SSD drive is awfully expensive. For the same money you get 20 2TB hard drives.
gnasher729 is offline   0 Reply With Quote
Old Dec 31, 2012, 01:33 PM   #16
phrehdd
macrumors 65816
 
Join Date: Oct 2008
Hard drives are a poor solution as failure rates are hit and miss. One brand x drive could last way beyond its expected life and the same brand x drive could fail the next day.

I highly suggest you look at media discs.

There are a few sites you can explore via Internet that will explain the differences between the discs, the type of materials and so forth. In the past, there were CD discs that were made strictly for archival purposes. They were made different than the typical discs and certainly cost a bit more. I would suggest you check into DVD discs and possibly blu ray (though I suspect the latter has not come as far enough along to be truly archival).

It has been a long while since I have looked at archival discs but here is an old site you can start with (sorry its not a plug for a particular product but it does reference them }- adterrasperaspera.com .

I am sure many will argue that drives are just fine. If you insist on this route, I would, as I would with discs, make multiple copies per a volume and also keep them at different locations.

Best to you.
phrehdd is offline   1 Reply With Quote
Old Dec 31, 2012, 01:35 PM   #17
rfle500
macrumors newbie
 
Join Date: Sep 2012
Long term storage of data is a particularly challenging problem. Most storage systems are designed with a ten year life span, including hard drives, ssd, DVDR etc. I actually research magnetic materials so can definitively help with how long they last and why. The data is stored on a number of magnetic grains which are designed to retain their written state for a period of around ten years. Thermal activation leads to a loss of this information over time, and so the data will certainly not be there forever. For hard disks a more common problem is the mechanical arm and head wearing out, which is typically on the three-five year timeframe. In industrial settings archives are generally stored on magnetic tape (digital) which are much simpler in operation and likely to age much better. I think they are also specified with 20+ year lifetimes but I am not sure. However, they are also much more expensive than disks.

Practically I would use a simple RAID 5/6 set up of multiple disks which I would completely reformat every 2-3 years. This covers the degradation of data with time and also the possibility of a failing disk, but obviously not fire/theft. Regarding other technologies SSDs are unproven and certainly higher capacity versions are more likely to degrade over time due to smaller cell sizes. DVDs are subject to rot, particularly DVDRs as the materials used tend to react with the atmosphere overtime, making the data unreadable. This probably also applies to BluRay discs as well.

1) Do hard drives degrade over time similar to the was that Hi8 tapes do?

Yes.

2) Each time I copy one hard drive to another, is some quality of the video/data lost?

HDDs include advanced error correction with corrected errors typically less than 10-14

3) with what frequency should I be backing up the hard drives-- every 5 years?

That's a good idea, more frequently is better

4) Is an enterprise class hard drive recommended for this. My cursory research suggests that these are muh higher quality drives than for example a Lacie external hard drive.

If it is for archive purposes then there is little advantage, but multiple disks in a RAID setup is a good idea

5) which external hard drive(and manufacturer) would you recommend. I need have about 2TB of data

I always liked Samsung drives, but WD have always been good for us

6) are SSD drives optimal or are these really not a proven technology yet[/QUOTE]

Very expensive as archive storage, and unproven for extended periods of time.
rfle500 is offline   0 Reply With Quote
Old Dec 31, 2012, 02:08 PM   #18
RedTomato
macrumors 68040
 
RedTomato's Avatar
 
Join Date: Mar 2005
Location: .. London ..
Quote:
Originally Posted by rfle500 View Post
Practically I would use a simple RAID 5/6 set up of multiple disks
...
If it is for archive purposes then there is little advantage, but multiple disks in a RAID setup is a good idea
I have no argument with the rest of your post, but I have to ask - have you ever actually worked with a RAID 5/6 set-up?

I have, and it is not pretty. RAID 5/6 is complex and answers a specific enterprise need. It is NOT for home use for the non-technical. You are introducing multiple points of failure. The OP is clearly non-technical, and looking after a RAID setup, especially after not having used it for two years, when you've forgotten the details of how you set it up, is full of traps. I've repaired RAID systems that have silently buggered up through user error, and it is not pretty and not for the faint of heart.

My advice? Keep it simple, cheap, and replicated. The OP says his data is 2TB. That's a small HDD. As a MINIMUM, keep 3 copies of the data in 2 different places. The original tapes form one copy, though as tape readers are becoming hard to find, it might be wise to not count on that.

Purchase 3 HDDs with 5 year warranties, minimum of 2 TB each, though 3 TB is cheap nowadays. USB3 as mentioned above is fine. Record a copy on each HDD. Give thought as to what format to store the video in - and store copies of applications and even operating systems that are able to read the video i.e. an install copy of Win 7 (or 8), an up to date Linux, OSX 10.8. At least 2 OS installers. And several apps that can show the video.

Consider buying two small computers (e.g. mac mini, or PC equivalent - though mac minis seem to last almost forever; my old workplace is still happily using a 7-year old mac mini G4 as a server) that can show the video, setting them up to work properly, then just storing them with the HDDs.

Two copies can be at your house or your office. The third copy needs to be in a physically different place (in case of fire or burglary) - across town or in a different city.

Also consider cloud storage. At the moment, 2TB is quite expensive to put on the cloud (though I hear Amazon Glacier is quite cheap). Prices will fall. That's a useful fourth copy in a non-physical place.

http://aws.amazon.com/glacier/

Budget for testing and replacing the HDDs every few years. And the computers too, though they will last longer.

In 15-20 years time, storage will be completely different. A 3TB USB3 HDD will seem as quaint and limited as an old typewriter ribbon - but with no typewriter to put it in.
RedTomato is offline   0 Reply With Quote
Old Dec 31, 2012, 02:27 PM   #19
rfle500
macrumors newbie
 
Join Date: Sep 2012
Quote:
Originally Posted by RedTomato View Post
I have no argument with the rest of your post, but I have to ask - have you ever actually worked with a RAID 5/6 set-up?
I have - at least the software RAID via Linux variety. However I appreciate your point and agree with your suggestions. In fact you can get simple plug-in sata USB adapters.

http://www.amazon.co.uk/dp/B0033AF5WW/

Just be sure to store your drives somewhere cool and dry.
rfle500 is offline   0 Reply With Quote
Old Jan 4, 2013, 07:00 AM   #20
Arran
macrumors 68000
 
Join Date: Mar 2008
Location: Atlanta, USA
Redundant copies on multiple hard drives is good, but use different drives from different manufacturers. Identical drives might share a manufacturing or design defect. Diversity is good for survival.

Personally, I have copies of my home videos on four different drives. Three of them are 3.5" drives (imac internal, time capsule and an external drive that only gets powered up once a year to check it's still readable). I recently added a 2.5" "self-powered" drive to the mix. It also gets powered up rarely.
__________________
When the world zigs, zag
Arran is offline   0 Reply With Quote
Old Feb 12, 2013, 02:40 AM   #21
Larry-K
macrumors 6502a
 
Join Date: Jun 2011
I've got a twenty year old hard drive sitting right here.

120MB, SCSI and it weighs about 5 pounds, doubt I could even read it, even if it would spin.

Go with something solid state that uses a widely accepted interface.

Call me in 20 Years, ask for the dead guy.
Larry-K is offline   0 Reply With Quote
Old Feb 12, 2013, 05:09 AM   #22
n8mac
macrumors 6502
 
Join Date: Jun 2006
Location: Ohio
Looking back 16 years ago when I was saving my high school art files, I had 2 choices available to me: saving to the hard drive and saving to 3.5" floppy disks. We had no CD burners because they were too expensive, slow, and untested. Then when leaving school I copied all files to a stack of floppies and kept them a few years until I got my first iMac, which had no floppy drive or burner . So I used work to get the files onto CDs, then put them in my iMac.

Since then with each Mac I used I just copy all files to my internal HDDs, then back up all my files on an external HDD. Two things still hold true: You will always be adding new files requiring more space and thus require you to copy your files to something bigger, and you need to keep copying your files to the latest medium you can afford. You can't expect anything you copy to now to still be usable 20+ years down the road. Some of those CDs I burned work, and some are totally unreadable. Now the latest iMac has no optical drive.

Just think about 20+ years from now what will still be relevant and able to store your petabytes worth of data? Those DVD/Bluray disks will be coasters and SATA interfaces will be history.
n8mac is offline   0 Reply With Quote
Old Feb 12, 2013, 08:32 AM   #23
utekineir
macrumors 6502
 
Join Date: Feb 2008
in 20 -40 years will you still have an usb/sata port?
utekineir is offline   0 Reply With Quote
Old Feb 16, 2013, 02:21 PM   #24
monokakata
macrumors 65816
 
monokakata's Avatar
 
Join Date: May 2008
Location: Hilo, Hawai'i
Here's the best solution:

http://www.the-scientist.com/?articl...-Here-to-Stay/

The authors have a sense of humor: "[DNA has] a proven track record as an information bearer."

I've linked to a news article, which cites the original (which I got around to reading today).

Here's what they encoded as a test:

" . . . all 154 of Shakespeare's sonnets (ASCII text), a classic scientific paper (PDF format), a medium-resolution colour photograph of the European Bioinformatics Institute (JPEG 2000 format), a 26-s excerpt from Martin Luther King's 1963 'I have a dream' speech (MP3 format) and a Huffman code used in this study to convert bytes to base-3 digits (ASCII text), giving a total of 757,051 bytes . . . ."

They encoded it in DNA, transported the DNA to another location, and decoded it with 100% accuracy.

Cost? The encoding costs are estimated at $12,400 MB (-1) for encoding and storage, and $220 MB (-1) for decoding.

So it's not cheap, but it's long-lasting and (unlike other solutions proposed here) isn't likely to become obsolete. The authors write, "As DNA is the basis of life on Earth, methods for manipulating, storing and reading it will remain the subject of continual technological innovation."

Also, DNA storage requires no active maintenance other than a cold, dry and dark environment. They suggest that DNA storage may be cost-effective for archives of "several megabytes with a ~600-5,000 yr horizon . . . ."

See? Perfect. 5,000 years ought to be enough.

Indeed they are serious in this, and if anyone reading this thread finds it interesting, he or she should take a look at the original article.
monokakata is offline   0 Reply With Quote
Old Feb 16, 2013, 03:22 PM   #25
phrehdd
macrumors 65816
 
Join Date: Oct 2008
So what have we all seen posted here -

Mechanical drives can fail
Tape backup can fail
Blu ray disc can fail
DVD disc can fail
CD disc can fail
Tape backup player/recorders can fail
Blu ray, DVD and CD burner/players can fail
SDD can fail
and so on....

It seems that one should not count on anything to last 20-30 years (though I honestly do believe there are disc media that fulfill the purpose quite well). Instead, one should have

multiple backups
test the backups
in case of failure of backup/archive, replace from other backup/archive
be prepared to transfer to newer technologies as needed
keep backups in more than one location

Testing archived information is as important as the archive themselves. No point in storing something only to find down the line it is not functioning properly. This is similar to a fail over test.
phrehdd is offline   1 Reply With Quote

Reply
MacRumors Forums > Apple Hardware > Buying Tips and Advice

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Similar Threads
thread Thread Starter Forum Replies Last Post
Imac with Fusion Hard Drive- HD Setup for video editing bimmeracer Digital Video 0 Jan 28, 2014 03:19 AM
WD Green Hard Drive for Video Editing and Storage ShermDog Digital Video 3 Nov 19, 2013 07:49 PM
Erasing Hard Drive with No Working Video Card three2won MacBook Pro 3 May 19, 2013 12:49 PM
Hard drive setup for video editing - help needed! yongy Digital Video 5 Jan 11, 2013 02:20 PM

Forum Jump

All times are GMT -5. The time now is 02:10 AM.

Mac Rumors | Mac | iPhone | iPhone Game Reviews | iPhone Apps

Mobile Version | Fixed | Fluid | Fluid HD
Copyright 2002-2013, MacRumors.com, LLC