Need DAS/NAS/Storage Reccomendations

[weezin]

I am a photographer who is sick of worrying about his files and dealing with a crappy, unreliable backup solution.

I currently have about 3TB of photos, and it's growing every day. Currently, I have My "working" files on my internal drive, a backup of ALL of my photos on a 4TB drive, and another copy of ALL of my photos split between two other drives. I also backup some of my files to Crashplan. It's confusing and frustrating. I need a flowchart to keep track of my stuff.

SO... I think something like a NAS/DAS is what I need. I know next to nothing about RAID, NAS or any other stuff like that. Easier is better.

I want to have a backup solution that I can set and not have to think about that will keep all of my files safe.

What would you recommend?



----------

In my initial research, it seems that something like a Drobo 5D would be perfect for me. 5 bays, redundancy, lots of storage if I want, and a "set it and forget it" mentality. However, I'm a bit turned off by its high price and the apparent history that Drobo has regarding reliability and customer service...

 

[flynz4]

I am a photographer who is sick of worrying about his files and dealing with a crappy, unreliable backup solution.

I currently have about 3TB of photos, and it's growing every day. Currently, I have My "working" files on my internal drive, a backup of ALL of my photos on a 4TB drive, and another copy of ALL of my photos split between two other drives. I also backup some of my files to Crashplan. It's confusing and frustrating. I need a flowchart to keep track of my stuff.

SO... I think something like a NAS/DAS is what I need. I know next to nothing about RAID, NAS or any other stuff like that. Easier is better.

I want to have a backup solution that I can set and not have to think about that will keep all of my files safe.

What would you recommend?



----------

In my initial research, it seems that something like a Drobo 5D would be perfect for me. 5 bays, redundancy, lots of storage if I want, and a "set it and forget it" mentality. However, I'm a bit turned off by its high price and the apparent history that Drobo has regarding reliability and customer service...

Moving your pictures to a NAS (RAID or not)... is just making another independent copy of your photos. RAID is not backup. Do not consider that RAID improves reliability over a single drive. Yes... in some way it helps, and in other ways, it creates new single points of failure.

Your best option is to keep your photos in one place... and double (or more) backup your photos to other destinations. I strongly suggest using automated methods to perform the backups.

I personally use:

1. Time Machine/Time Capsule for hour local backups
2. Crashplan+ for offsite backups (every 15 minutes)

The above give me dual independent backups, fully automated, using two different backup programs, to two different destinations. Lots of good things going on here.

Because I care about data integrity a lot... I also additionally clone my data. Note, that I do not consider "clones" to be a true backup (because of limited versioning)... but they are still valuable once you have a good backup system in place. Personally I clone as follows:

• Nightly clone of all of my personal media (incl photos)... to a Thunderbolt attached portable drive
• Manual clones to a pair of cheap USB HDDs... the latest of which is always manually stored offsite

The first clone gives me a working copy of my libraries... just in case my primary computer crashes... I can take my latest library and move to a new computer without delay.

The second pair of clones give me an offsite copy of all my personal media. So even if my house burned down and my cloud backup provider went out of business on the same day... I still have my data saved offsite (possibly slightly old versions).

BTW: I have a Pegasus DAS array that I do use as primary storage on my otherwise 100% SSD based iMac. I also have several NAS boxes that do RAID that I have used over the years. For the most part, they are turned off and collecting dust.

/Jim

 

[weezin]

Moving your pictures to a NAS (RAID or not)... is just making another independent copy of your photos. RAID is not backup. Do not consider that RAID improves reliability over a single drive. Yes... in some way it helps, and in other ways, it creates new single points of failure.

Your best option is to keep your photos in one place... and double (or more) backup your photos to other destinations. I strongly suggest using automated methods to perform the backups.

I personally use:

1. Time Machine/Time Capsule for hour local backups
2. Crashplan+ for offsite backups (every 15 minutes)

The above give me dual independent backups, fully automated, using two different backup programs, to two different destinations. Lots of good things going on here.

Because I care about data integrity a lot... I also additionally clone my data. Note, that I do not consider "clones" to be a true backup (because of limited versioning)... but they are still valuable once you have a good backup system in place. Personally I clone as follows:

• Nightly clone of all of my personal media (incl photos)... to a Thunderbolt attached portable drive
• Manual clones to a pair of cheap USB HDDs... the latest of which is always manually stored offsite

The first clone gives me a working copy of my libraries... just in case my primary computer crashes... I can take my latest library and move to a new computer without delay.

The second pair of clones give me an offsite copy of all my personal media. So even if my house burned down and my cloud backup provider went out of business on the same day... I still have my data saved offsite (possibly slightly old versions).

BTW: I have a Pegasus DAS array that I do use as primary storage on my otherwise 100% SSD based iMac. I also have several NAS boxes that do RAID that I have used over the years. For the most part, they are turned off and collecting dust.

/Jim

Thanks for the info.

Part of the problem I'm running in to is the fact that my 4tb drive is going to be full soon. That doesn't leave me very many options. I was thinking something like a Drobo 5D...that would give me up to ~12gb of storage (RAID 5 ish) and would also give me the benefit of having multiple copies in one place, but on separate hard drives.

I do use Crashplan+, but the problem is that I have SO MANY times (~3.3TB) that it will take forever and a day (like a year) to backup to that spot.

Obviously I know the ins and outs of my storage/backup situation better than anyone, and it looks like a Drobo 5D or other multi-hd RAID setups would be the way to go in combination with crashplan local backup or time machine. But then again, I know next to nothing about this.

 

[AppleDApp]

Thanks for the info.

Obviously I know the ins and outs of my storage/backup situation better than anyone, and it looks like a Drobo 5D or other multi-hd RAID setups would be the way to go in combination with crashplan local backup or time machine. But then again, I know next to nothing about this.

Dont do RAID for backup.

 

[weezin]

Dont do RAID for backup.

Why not? I hear about people using 4 - 5 drive bays with RAID 5 all the time for backup. Is there a specific reason?

 

[chfilm]

I have a drobo 5d with 5x1tb drives in there and am thinking of selling it..
I think the drobo is a nice solution not just for backups but also for foto editing.
It's pretty fast compared to old Drobos. I actually get up to 350mb/s read.

 

[weezin]

I have a drobo 5d with 5x1tb drives in there and am thinking of selling it..
I think the drobo is a nice solution not just for backups but also for foto editing.
It's pretty fast compared to old Drobos. I actually get up to 350mb/s read.

Nice! I'd probably go for a combination of 2, 3 and 4TB drives (since that's what I already have...).

 

[blueroom]

I use a Synology NAS, excellent.

RAID is not a backup, you can backup a RAID to another drive or the cloud.

 

[flynz4]

Why not? I hear about people using 4 - 5 drive bays with RAID 5 all the time for backup. Is there a specific reason?

RAID is indeed valuable when you need a larger volume than you can get with a single spindle drive. So if you really meet a volume larger than 4TB, the by all means go ahead and get a RAID array.

The recommendation still stands to consider the array to have the same likelihood of losing data as a single spindle drive. It is not necessarily the case... but as I said earlier... on one hand, you get some reliability improvement by having your data spread redundantly across several drives... but RAID also introduces new single point of failure mechanisms that are unique to RAID. Hence... there are pluses and minuses... that trade off against each other.

The other traditional use for RAID was to improve IOPs & BW performance. However, with inexpensive SSDs, there is about no reason to use RAID for performance. Just go pure SSD.

With the huge amount of data that you have... you may want to consider if it all really needs to be backed up. For example... I ripped my entire DVD collection and store the data digitally. There is no reason why I need to back up all that data... because it all exists already out in the world, and is trivially re-attainable. I many ways, I am not sure why I even keep it anymore... given that I can stream it all for "essentially free" whenever I want to view it. In other words... I am keeping data that has essentially no value. I do not bother backing that up to Crashplan at all... there simply is no need. Quite frankly, I am not sure why I just don't delete it all.

OTOH... my personal media. Photos, Camcorder videos, FCPX projects, etc... are extremely valuable to me. My Aperture library is pretty big (400GB)... but that takes relatively little time to back up to Crashplan. My entire set of data backed up to CP is currently 1.5 TB. What I did was pay the $125 to "seed" the backup to Crashplan... and in a couple of days, 1.2TB (the most I could fit on the drive) was loaded into my account. The remaining 300GB of data uploaded in less than a week.

/Jim

 

[Ifti]

Although RAID is not a backup in itself, it is another layer of protection in the event of drive failure.
I have been using a Drobo 5D for quite a while now - mainly for video editing, and also as a backup of the info on my server, and its always been spot on.

 

[AppleDApp]

Although RAID is not a backup in itself, it is another layer of protection in the event of drive failure.
I have been using a Drobo 5D for quite a while now - mainly for video editing, and also as a backup of the info on my server, and its always been spot on.

YouTube: video

How do you feel about the fact that it is proprietary?

 

[maflynn]

How do you feel about the fact that it is proprietary?

I'm not defending drobo, (even though I have one), but I wanted to point out a simple observation. People bring up the fact that drobo uses a proprietary RAID implementation. Granted, I too would prefer the standard vanilla RAID options but with that said, when other NAS discussions occur and the recommendation for Synology is made, nothing is mentioned about them using a proprietary RAID as well.

As I said, I would much prefer a standard raid, but Drobo's proprietary RAID is rock solid. If you're going to knock the company then I'd say their poor customer support and/or some of the hardware failures that occurred early on. I think the 5D/5N and my Mini seem to be fairing better.

 

[torana355]

As others have said a RAID is not a sufficient backup, it only gives you redundancy. I have a Synology 4 drive setup as Raid 10 which i backup to two external 4tb drives. I personally would not touch another drobo with a ten foot pole, ive seen way too many of them fail including the latest models, they have incredibly slow throughput and the proprietary RAID system that they use is hard to repair if the array fails. We use Synology RAIDs at work and have never had an issue with them

 

[weezin]

RAID is indeed valuable when you need a larger volume than you can get with a single spindle drive. So if you really meet a volume larger than 4TB, the by all means go ahead and get a RAID array.

The recommendation still stands to consider the array to have the same likelihood of losing data as a single spindle drive. It is not necessarily the case... but as I said earlier... on one hand, you get some reliability improvement by having your data spread redundantly across several drives... but RAID also introduces new single point of failure mechanisms that are unique to RAID. Hence... there are pluses and minuses... that trade off against each other.

The other traditional use for RAID was to improve IOPs & BW performance. However, with inexpensive SSDs, there is about no reason to use RAID for performance. Just go pure SSD.

With the huge amount of data that you have... you may want to consider if it all really needs to be backed up. For example... I ripped my entire DVD collection and store the data digitally. There is no reason why I need to back up all that data... because it all exists already out in the world, and is trivially re-attainable. I many ways, I am not sure why I even keep it anymore... given that I can stream it all for "essentially free" whenever I want to view it. In other words... I am keeping data that has essentially no value. I do not bother backing that up to Crashplan at all... there simply is no need. Quite frankly, I am not sure why I just don't delete it all.

OTOH... my personal media. Photos, Camcorder videos, FCPX projects, etc... are extremely valuable to me. My Aperture library is pretty big (400GB)... but that takes relatively little time to back up to Crashplan. My entire set of data backed up to CP is currently 1.5 TB. What I did was pay the $125 to "seed" the backup to Crashplan... and in a couple of days, 1.2TB (the most I could fit on the drive) was loaded into my account. The remaining 300GB of data uploaded in less than a week.

/Jim

Jim...thanks a ton for the response. I understand what you are saying for sure about a RAID array fort backup.

For me, all of that 3TB+ of data is "personal" data...ie photos and videos. I'm an avid photographer, mostly for myself, but occasionally for work, and the RAW and TIFF scans take up a ton of space. Because of this, I need to back them up (they're irreplaceable).

Seeding the data to crashplan seems like it would be a great option for me. When you consider that RAID arrays are $600+, $125 seems reasonable.

If I were to do that, I would still need a local solution for my data, and because I'm approaching my 4TB limit, that solution would probably have to be some sort of a RAID array.

----------

Although RAID is not a backup in itself, it is another layer of protection in the event of drive failure.
I have been using a Drobo 5D for quite a while now - mainly for video editing, and also as a backup of the info on my server, and its always been spot on.

The 5D/5N do look really nice and actually look reasonably priced compared to other RAID solutions (like Synology). They also seem to give you lots of options with regard to drive size, etc. It seems the most user friendly to me from what I've read.

----------

I'm not defending drobo, (even though I have one), but I wanted to point out a simple observation. People bring up the fact that drobo uses a proprietary RAID implementation. Granted, I too would prefer the standard vanilla RAID options but with that said, when other NAS discussions occur and the recommendation for Synology is made, nothing is mentioned about them using a proprietary RAID as well.

As I said, I would much prefer a standard raid, but Drobo's proprietary RAID is rock solid. If you're going to knock the company then I'd say their poor customer support and/or some of the hardware failures that occurred early on. I think the 5D/5N and my Mini seem to be fairing better.

I've noticed this also. People complain about the proprietary RAID in the Drobo, and then recommend the Synology (which is also proprietary!). Why is this?

 

[mikepj]

I've noticed this also. People complain about the proprietary RAID in the Drobo, and then recommend the Synology (which is also proprietary!). Why is this?

The difference is that with Synology boxes, you have a choice of which RAID level you want to run. The default is their proprietary hybrid RAID, but Synology NAS devices are just specialized computers running Linux so you have the option to use RAID 0, 1, 5, 6, 10, or any other supported Linux RAID level (and you can configure this through their standard setup interface). If you use one of the standard RAID levels and the Synology box fails, then theoretically you should be able to move the hard drives to any other Linux computer and still see your files. Linux software RAID is pretty reliable, because so many people are using it on so many computers both at the enterprise and consumer level. I've been using Linux software RAID on countless servers for the past 10-15 years, and have never seen a volume failure resulting in data loss (even though I've had plenty of drives fail).

With Drobo, your only option is their proprietary RAID. If the box fails, you have to buy another Drobo. If their RAID software fails, then you are completely out of luck. While Linux RAID is open source and thus presumably more stable (more eyes looking at it for problems), Drobo's proprietary RAID is closed-source and there is no way to verify how stable their implementation is.

 

[FireWire2]

As mikepj was saying, that is true, addition to it both also use software RAID.

To safely back up, you need multiple copies.

Use both NAS and DAS if you will
DAS you can use
Drobo - TB/USB3.0
http://www.amazon.com/Drobo-5-bay-Storage-Array-Thunderbolt/dp/B008MH1JRQ
Sans Digital - eSATA/USB3.0
http://www.amazon.com/Sans-Digital-TowerRAID-TR5UT-Hardware/dp/B004WNLPH8
DATOptic - TB/eSATA/USB3.0
http://www.datoptic.com/ec/dual-thunderbolt-raid-with-5x-sata-bay-for-mac-and-windows.html

As for NAS you can use DATOptic, Drobo, Netgear, Qnap....

But the only NAS that has HW raid is DATOptic eBOX-N

 

[zuluboy]

Sorry to butt in

I have similar problem. I have ordered new Mac Pro as my 2008 Mac pro now battles to process the very large Powerpoint files I work with (some are 150 pages plus with quarterly data going back 10 years & each quarter may have 2-5 data points. Sadly Keynote just can't handle this and most of my clients Banks, Accounting firms, Lawyers etc run windows ).

I plan to keep the new Mac Pro internal drive for the OS only - no data) and use my Old MacPro as as the external disk drive (it has 3x 1TB and another with the OS) the link will be via ethernet and/or USB (but the old machine only has USB2 so that will limit data transfer speeds).

I have 2 questions 1) is Ethernet (or USB) fast enough to handle the Powerpoint files when am working on them (i.e. no latency I think is the potential issue) or 2) should I invest in either a NAS or DAS and keep all data on that with the old Mac Pro as just another backup (I could also sell it) (I use Carbonate for offsite backup but wonder if Crashplan is better as it offers continuous backup.)

One other point (apologies for darting around) relates to the spec for the new Mac Pro. I discussed 2 specs with Apple - Option 1 starts with the basic 4 core model but then ungraded to 6 core and 16GB RAM (vs original 12GB). Option 2 is simply to go for the basic 6 core model which already has 16GB of Ram. There is a difference of c $350 ish which is because option 2 has a better graphics card with an additional 1Gb of RAM. Am I doing the right thing. The idea is to enable me to handle these large Powerpoint files and keep around 10 applications open. Apologies for this red herring but any help appreciated.

 

[maflynn]

I've noticed this also. People complain about the proprietary RAID in the Drobo, and then recommend the Synology (which is also proprietary!). Why is this?

I'd say the popularity of the Synology is party the cause. The Synology is a good NAS, no question, but I'm happy with my Mini DAS. I also own a qnap NAS but find the drobo DAS to better fit my usage patterns.

----------

The difference is that with Synology boxes, you have a choice of which RAID level you want to run.

Agreed, and I'll not dispute that with the synology you have more choices (which is a good thing) but I also suspect that the majority of people who buy the synology use the default proprietary RAID.

I'm not against synology, its a good NAS, I was just making an observation

 

[monsieurpaul]

IMHO, the key point is to have an automatic backup. That's why people are always talking about RAID in this kind of thread because that's what RAID looks like (you put your stuff on a drive and you won't have to worry anymore about it) even though RAID is not a backup solution, and you don't need RAID to have automatic backup.

Considering the size of photos to backup it could be interesting to know which software is used (Aperture, Lightroom ?) with what kind of library (Referenced, Managed ?) as it can drive other solutions for backup. There are very interesting thread about photos workflow (including backup) in the Digital Photography section of this forum.

Personnally, I use Aperture on a Macbook pro with a much smaller library (around 160 GB) and the master files are referenced e.g. they are not included in the Aperture library file but are stored on separate folders. I also own an Synology DS212j, with no RAID but external HDs plugged to the Synology in USB. The backup process is as follows:

- The photos are imported from the camera to the Macbook pro
- The folder where the photos are is synced on the Synology. I use BitTorrent Sync which is a magical piece of software: it's free, fast, reliable, the client is easily installed on a Synology and it works wherever you are on your local network or across the world in a hotel room with WiFi.
- Then, the Synology runs 2 backup jobs (among others) every day: one local that copy the folder on one of the external drive plugged to it, the other on an Amazon S3 account. I'm using a "Glacier" type of storage so it costs me around 1 $ a month. BUt for larger size it could be more interesting to look into Crahsplan+ solution.

That way, the backups are full automatic, I have a local copy on my Synology in a matter of minutes and I have one additionnal local copy + one copy in the cloud in less than 24 hours.

The Synology DS212j is one of the cheaper model and I use it also to share videos and music across the house to various devices such as Sonos or WD TV video streamer, to share folders between all the computers in the house, to provide airprint for the iPhones and iPad in the house (I just plugged an old HP laserjet on USB, checked a box and it worked), to provide a personal cloud storage and to run a git server. This is one of the best piece of hardware/software I have ever bought.

 

[weezin]

Thanks for the replies everyone...much appreciated.

I've decided to give a Drobo 5D a try. It seems to hit all of my targets for what I think a NAS/DAS should be. The ability to hot swap different sized drives (than what was originally in there) is an incredible feature over other NAS/DAS systems alone (for me). Provides easy expand-ability it seems.

So here is what i'm thinking my backup situation will look like:

My working directory will be on my internal drive.
The internal drive will be automatically backed up to my Drobo via Crashplan.
The Drobo will house ALL of my photos natively.
The Drobo will be backed up to Crashplan servers.
The Drobo will also be backed up periodically to 1 or 2 external hard drives that will be kept at my work.

When images are imported to Lightroom from my camera, a copy will go into my working directory on my internal drive and a second copy will go onto the Drobo.

This way, I have at least 3 sets of a file in any one place, and all of them will be "versioned".

Does this make sense?

 

[FrozenDarkness]

I personally LOVE synology products so I would suggest any of their bays for a reliable machine.

 

[danny_w]

I personally LOVE synology products so I would suggest any of their bays for a reliable machine.

Unfortunately all of the Synology boxes are NAS not DAS. I really need DAS since ATV cannot read from a NAS and data must be streamed from the NAS to the computer (over wifi) and then back to the ATV ( again over wifi) since my computer is upstairs and wireless.

 

[FrozenDarkness]

Unfortunately all of the Synology boxes are NAS not DAS. I really need DAS since ATV cannot read from a NAS and data must be streamed from the NAS to the computer (over wifi) and then back to the ATV ( again over wifi) since my computer is upstairs and wireless.

ATV2 has XBMC capability

 

[danny_w]

ATV2 has XBMC capability

Only if you jailbreak and my wife is used to the ATV menus now so that's a no-go. Is there still no NAS out there that can do ATV streaming? I know that you used to could run iTunes on Windows Home Server.

 

[FrozenDarkness]

Only if you jailbreak and my wife is used to the ATV menus now so that's a no-go. Is there still no NAS out there that can do ATV streaming? I know that you used to could run iTunes on Windows Home Server.

Synology can run itunes server for music. Thats bout it.