How do you manage media storage?

[BJB Productions]

Hi! I'm a photographer / filmmaker. Up until now, I've been buying OWC external drives for all my media. Now that 4k is a thing and RAW files are so massive, I'm using space a LOT faster. I'm looking for a simple, affordable solution to archiving data.

I was thinking of going the route of purchasing a hard drive dock and then purchasing bare 3.5" drives which will also be backed up to BackBlaze. I still have a working SSD drive, so this is purely for data archival.

Thoughts?

 

[ZapNZs]

What kind of space do you need?
What kind of budget are you on?
How much speed do you want? (some of the faster 7200 RPM HDDs in the right RAID config can approach some SATA SSD speeds )
Are you open to a RAID setup using four 3.5-inch HDDs or a mirrored RAID setup using two very high-capacity 3.5-inch HDDs?

I'm very biased toward the HGST Ultra Star line, Sonnet, and AKiTiO.

 

[BJB Productions]

What kind of space do you need?
What kind of budget are you on?
How much speed do you want? (some of the faster 7200 RPM HDDs in the right RAID config can approach some SATA SSD speeds )
Are you open to a RAID setup using four 3.5-inch HDDs or a mirrored RAID setup using two very high-capacity 3.5-inch HDDs?

I'm very biased toward the HGST Ultra Star line, Sonnet, and AKiTiO.

Budget: as low as possible. Couple hundred at most.
Speed: 7200rpm speed is just fine. This is just data archive.
I'm open to options.

 

[ZapNZs]

How much size do you want?
Do you want protection against hardware failure?

IMHO, and while I am anything but an expert on the subject...If you want pure economy without redundancy, you could go for the Mobius 2-bay drive and two 4 TB WD Reds in a RAID 0, giving you 8 TB of storage - but, should one drive fail, you either have to bite the bullet for several hundred-to-several thousand dollar data recovery, or accept that all of the data on both drives will be lost. Alternatively, you could use the same products in a RAID 1 configuration, giving you 4 TB of storage, and redundancy in that each drive would automatically store the same files as the other (arguably much more desirable if you wish to protect against data loss from hardware failure [but not corruption].) Also alternatively, you could use the drives in the 2-bay enclosure to manually back up the files yourself to maintain one version of your most important files on each drive. You could do any of these options for around $400.

If you wanted more reliability, a longer warranty, and/or longer anticipated service lives, the HGST Ultra Star is my biased choice, but it would be pricier (about $100 more per 4TB drive over the WD Red.) I absolutely love these drives and swear by their reliability, ability to withstand 24-7 usage, and long service lives - so I am biased here. Like many other 7200 RPM 3.5-inch enterprise drives, they are extremely fast for HDDs (in many cases, over 200 MB/s read/write) and rather loud.

If you wanted expandability, one of the 5-bay enclosures might be ideal. Even if you do not use RAID at all (or use soft 1/0) for now, the 5-bay HW RAID enclosure would give you the ability to implement hardware RAID 3/5/10 in the future, should you desire, and the independent 5-bay means you could implement a software RAID 1/0 and use another disk independently to store the same files using a different means (so you can account for both hardware failure and software corruption.) Especially if you plan to continue to accumulate 4k video, a 5-bay enclosure gives you a lot of expandability where you could use your existing enclosure and just add additional drives to it (versus having to purchase another enclosure.)



Possible Enclosures?
USB/eSATA/FW 2-bay, and capable of hardware RAID0/RAID1
USB 2-bay, and capable of hardware RAID0/RAID1 (this is a relatively new budget line product and I imagine it is not built to the same quality as their pricier dual and quad bay enclosures?)
USB/eSATA/FW800 and capable of hardware RAID0/1/5/10
Thunderbolt 2 & 5-bay, and capable of soft RAID

Possible Drives?
6TB Desk Star
4TB WD Red
2 TB Ultra Star
4 TB Ultra Star

 

[Outrigger]

Budget: as low as possible. Couple hundred at most.
Speed: 7200rpm speed is just fine. This is just data archive.
I'm open to options.

not sure how much a couple of hundred will get you aside from a bare harddrive.

 

[BJB Productions]

not sure how much a couple of hundred will get you aside from a bare harddrive.

Right. So what's the advantage of buying a 4 or 5 bay enclosure vs just a hard drive dock if I'm not raiding? I was thinking of just using back blaze to back up each hard drive.

 

[MrX8503]

Right. So what's the advantage of buying a 4 or 5 bay enclosure vs just a hard drive dock if I'm not raiding? I was thinking of just using back blaze to back up each hard drive.

Multiple bays allow you to have redundant backups. If you can only afford 1 hard drive then just get the biggest external hard drive you can find. However, you'll only have 1 local backup. Your second backup will be from the cloud.

My backup setup is I clone my entire machine to a single external weekly. I clone my home folder to the same drive daily. Finally, I have time machine running on the whole machine on a second hard drive.

Time machine is for accidental deletions and the weekly clones are for checking my data integrity.

 

[ZapNZs]

FWIW, I also use multi-modal backups as such because it seems to be difficult to account for both hard disk failure and corruption through a single method only - I'm concerned over both failure types given they both have the potential for data loss, and data loss for my usage at this point in time would be harmful to my productivity and income potential.

For me personally, the expandability of the multi-bay to handle increasingly large files I was working with, Thunderbolt connectivity to allow the use of external SSDs with TRIM enabled, the ability to implement a complete backup solution to address physical failure + corruption, and RAID expandability for future needs, was what I found attractive.

 

[BJB Productions]

I'm looking at just using a SSD or RAID as a working drive, then using Backblaze B2 for archive.

 

[MrX8503]

Given your budget, I wouldn't use RAID.

 

[ZapNZs]

You could get a single 6TB WD Red for around $220 or a 6TB HGST Ultra Star for around $330, which when combined with a cloud backup, would give you protection against data loss from hardware failure.

 

[priitv8]

I was thinking of just using back blaze to back up each hard drive.

That's also the solution I am currently using.
It works OK in many respects and Backblaze's service is also good for the money. The only problem, that will develop over time (as the number of your external disks grows) is, that BB will delete your external's backup from their cloud if drive has not been seen for over 30 days.
This makes you constantly rotate your drives on the computer you have set up as host to be backed up (BB pricing is per host, no space limitation other than the permanent visibility of drives).
Theoretically, there will come a point, when rotating disks becomes not possible in time window given (30 days).

 

[BJB Productions]

That's also the solution I am currently using.
It works OK in many respects and Backblaze's service is also good for the money. The only problem, that will develop over time (as the number of your external disks grows) is, that BB will delete your external's backup from their cloud if drive has not been seen for over 30 days.
This makes you constantly rotate your drives on the computer you have set up as host to be backed up (BB pricing is per host, no space limitation other than the permanent visibility of drives).
Theoretically, there will come a point, when rotating disks becomes not possible in time window given (30 days).

Right. Have you looked into B2? It's Blackblaze's new pro service. Looks awesome.

 

[sboychuck]

For something very inexpensive, you can try this, or this with two bays. I have a one bay and have an HGST 3.5" NAS Drive, 4TB, in it at 7200 rpm. It runs off of USB3. I use it for Time Machine backups, and also clone the hard drive at least once a month on a separate HGST NAS drive. It is very inexpensive and the clone is stored off site locally. USB3 is pretty quick. You can then also back-up to the cloud for even more redundancy. Just a thought...

 

[priitv8]

Right. Have you looked into B2? It's Blackblaze's new pro service. Looks awesome.

Hi. I have only briefly glanced over it.
Looks more like a business-oriented solution to me:
1) no desktop tool available, only Web-UI and partner-integration
2) pricing based on storage-space, with both up- and downloaded bytes being charged. 4:1 price penalty for download.

 

[BJB Productions]

Hi. I have only briefly glanced over it.
Looks more like a business-oriented solution to me:
1) no desktop tool available, only Web-UI and partner-integration
2) pricing based on storage-space, with both up- and downloaded bytes being charged. 4:1 price penalty for download.

Yes. I'm trying to compare it to Amazon Cloud or just the regular Backblaze service.

 

[priitv8]

4:1 price penalty for download.

That was my mistake.
Reality is this:
1GB upload = $0.39
1GB download = $0.24
It's only the initial upload, where 1GB = $0.06

 

[BJB Productions]

That was my mistake.
Reality is this:
1GB upload = $0.39
1GB download = $0.24
It's only the initial upload, where 1GB = $0.06

Yikes. That's somewhat steep. So should I just stick with hard drives and Backblaze's standard backup? I need some sort of cloud solution. Problem is, I have so many TBs of archival media it will take months for my backup to me complete with my bad ISP.

 

[BJB Productions]

Any other thoughts?

 

[BJB Productions]

I'm also worried about B2 because my cost will go up every year as I add more and more data. Eventually, I'll be to the point where I'm paying $1,000 a year for data. That's a long way off, but it still concerns me. Though I suppose you get what you pay for. What I like about it is that I can stop stacking up hard drives and not have to worry about having physical media around. The problem is, I never know when I'm going to need to download some data, which could get pricey.

 

[ZapNZs]

If you are on the road to paying that much for data, why not invest in your own off-site NAS setup? It sounds like you could save a fortune this way...

 

[BJB Productions]

If you are on the road to paying that much for data, why not invest in your own off-site NAS setup? It sounds like you could save a fortune this way...

Hmm, how would I go about setting this up? Sounds pricey and complicated.

 

[Fishrrman]

OP:
What are your storage needs?
Do you need 1tb? 2tb? 4tb? More?

Even around 4tb, I'd go with external backup drives stored at different locations (offsite).

I don't trust "cloud storage". I want my backups "reachable" right here on earth...

 

[BJB Productions]

OP:
What are your storage needs?
Do you need 1tb? 2tb? 4tb? More?

Even around 4tb, I'd go with external backup drives stored at different locations (offsite).

I don't trust "cloud storage". I want my backups "reachable" right here on earth...

I'm probably looking at 6-7tb of storage. The reason B2 intrigues me is it's easy to access all my data any time.

 

[Plett]

I use two HDD at any one time cloned with carbon copy cloaner and backed up with backblaze. I keep a second redundant copy on my commercial site for clients to be able to access and download/print at their leisure. Seems to work very well. When the HDDs fill I box the drive and stick in the closet and get two new drives, and repeat.
[doublepost=1497112726][/doublepost]One more thought, in my contracts I only agree to store data for 1 year from delivery (I do keep data longer, but don't make any promises to that end) The way I see it if a client wants photos archived they should sort that out.