Please advise: External RAW storage RAID

[MP3.1]

Hi all!

Would be awesome if you could help me out with this one . Planning on purchasing the following imac 5k: 4Ghz i7, 24Gb RAM, 256Gb SSD, R9 M395. Do mostly photo editing, some timelapse. I would like to set up a decent future proof RAW file storage & backup system taking the following into account:
- RAW's won't go onto the 256Gb SSD, Lightroom library will.
- The RAW files need to be stored in such a way that I won't experience/notice any editing lags through Lightroom.
- Apple time machine setup also a possibility. Does this work with RAID?
- Single user access
- my house has Gigabit ethernet.
- will also manually backup RAW files for off-site storage.
- budget around 600 Euros max (would be around $650 USD)
- Photo collection will be lower than 1Tb for the coming 5 years.

So I basically need fast external storage, doesn't have to be a lot, and backup capabilities (would be fine with RAID1).

What would your advice be? NAS? DAS RAID?

Thanks a lot for your advice!!

 

[toptoptopper]

Hi all!

Would be awesome if you could help me out with this one . Planning on purchasing the following imac 5k: 4Ghz i7, 24Gb RAM, 256Gb SSD, R9 M395. Do mostly photo editing, some timelapse. I would like to set up a decent future proof RAW file storage & backup system taking the following into account:
- RAW's won't go onto the 256Gb SSD, Lightroom library will.
- The RAW files need to be stored in such a way that I won't experience/notice any editing lags through Lightroom.
- Apple time machine setup also a possibility. Does this work with RAID?
- Single user access
- my house has Gigabit ethernet.
- will also manually backup RAW files for off-site storage.
- budget around 600 Euros max (would be around $650 USD)
- Photo collection will be lower than 1Tb for the coming 5 years.

So I basically need fast external storage, doesn't have to be a lot, and backup capabilities (would be fine with RAID1).

What would your advice be? NAS? DAS RAID?

Thanks a lot for your advice!!

I'm also very interested on people's thoughts on this - I've just ordered a very similar spec to above (and have a similar Lightroom workflow) and I'm not sure that my existing Western Digital USB3 drive (not raid) will give me the speed I'm hoping for - but Thunderbolt 2 seems to cost a huge amount, particularly for larger capacities. I'll be watching this with anticipation!

 

[AFEPPL]

Time machine wouldn't work well - you will see/feel a huge delay.
The drives used in them are slow and spend most of their lives asleep then waking up.

I'd go for a NAS device if you are do lots of random reads/writes you can go for SSDs to improve the performance by reducing latency however this would be beyond your budget. What are you wanting RAID for redundancy or increased throughput? I use synology and you can connect them directly to a cloud provider and run a schedule job to backup your local data.

 

[monokakata]

When I went from a classic Mac Pro to the retina iMac, I went all-in on TB2. I got the OWC 4 bay external, which is out of your price range. I had two 1 TB SSDs that had been in the Mac Pro. I put one in the OWC box, and when I edit D810 RAW files (typically in the 40 mb range) using LR CC, I'm not aware of any lag.

I also use a retina Macbook Pro and wanted a portable drive. I couldn't find an empty TB single-disk housing, so I bought a G-drive Mobile 2.5" TB unit, opened it up (yes, of course invalidating the warranty) and swapped out the spinning drive for the second 1TB SSD that I already had. That unit is also very fast. No lag whatsoever. I then bought an inexpensive USB3 enclosure and put the ex-G-drive spinner in it. If you did something like that, you could use the spinner as your TM drive, via USB. I'm not sure about doing this for 600 euros, but if SSD prices have fallen significantly, you might be able to do it.

I will say that doing the G-drive Mobile swap was a little tricky. The case is beautifully designed and it's not as though you can remove 6 or 8 screws and be done with it. Care is required.

 

[MP3.1]

Thanks all for your reply's! I read that NAS can be quite slow. I'm now thinking about DAS with 2x3Tb connected trough USB 3.0 and set it up as RAID(1). I'll get something faster when SSD prices drop..

 

[kohlson]

Thanks all for your reply's! I read that NAS can be quite slow. I'm now thinking about DAS with 2x3Tb connected trough USB 3.0 and set it up as RAID(1). I'll get something faster when SSD prices drop..

Just as a data point: I have a 256 GB SSD that I took from a 2011 MBA, and put it in an OWC Envoy USB3 enclosure. Using the Black Magic speed test, it writes from 290-310 MB/s, and reads from 310-350 MB/s. Generic mSata drives and adapters are pretty inexpensive, and newer drives are probably better performers.

 

[HDFan]

You might want to check out Oyen Digital's 5 Bay USB 3 Raid enclosure. It's inexpensive. $239 from vendor, I bought it for less used on Amazon directly from Oyen.. The cost with 5 3 TB drives @90 each giving you about 12 TB capacity would be $689. Way more than you need so you could buy smaller drives and be under budget.

USB 3 runs up to 640 Mbps (theoretical) so if you are just using one device on the port there's no need for the extra expense of thunderbolt. They spec it out at 375 MB/sec. My almost 3X more expensive Drobo 5D comes in at 253 MB/s write, 428 MB/s read.

I use it in a JBOD configuration rather than RAID 5 so can't confirm their numbers. But they have top of the line support which I have used a couple of times.

 

[BobertKennedy]

USB 3 runs up to 640 Mbps (theoretical) so if you are just using one device on the port there's no need for the extra expense of thunderbolt. They spec it out at 375 MB/sec. My almost 3X more expensive Drobo 5D comes in at 253 MB/s write, 428 MB/s read.

In reality, you will be lucky to see 400Mbps. There are a lot of factors involved: speed of the drive, drive cache size, operating system, load on USB controllers, etc.

 

[robgendreau]

I don't think you need RAID for speed for the RAWs. But you do need a bigger SSD in that machine.

Here's why: Lr will store the previews, preferably 1:1 previews, for all those photos, or at least the ones you're using lately. And with time lapses that could be a LOT of images. Lr will access the previews frequently, but the RAWs not so much. As a data point I've got a pretty small Lr catalog, less than 10k by a lot, including lots of older lower res images. And I only keep previews 30 days. But after a three week trip and importing relatively small (15MBish) RAWS I had about 100GB of previews on the boot SSD, and that doesn't even count the catalog and any smart previews.

And I dunno your camera situation, but 1TB seems small for five years, unless you're not gonna upgrade equipment or are better than most at culling. I've now got a pixel shift capable camera, and one RAW runs over 100MB.

I regularly move photos off the boot SSD to either an attached Thunderbolt SSD (didn't get it for that; was leftover equipment from another computer setup) or a HHD via USB 3. Because of how Lr handles the previews I don't really notice much diff in accessing images as long as the previews are up to date, since they are in the place that gives the fastest access: the boot drive.

 

[MP3.1]

I don't think you need RAID for speed for the RAWs. But you do need a bigger SSD in that machine.

Here's why: Lr will store the previews, preferably 1:1 previews, for all those photos, or at least the ones you're using lately. And with time lapses that could be a LOT of images. Lr will access the previews frequently, but the RAWs not so much. As a data point I've got a pretty small Lr catalog, less than 10k by a lot, including lots of older lower res images. And I only keep previews 30 days. But after a three week trip and importing relatively small (15MBish) RAWS I had about 100GB of previews on the boot SSD, and that doesn't even count the catalog and any smart previews.

And I dunno your camera situation, but 1TB seems small for five years, unless you're not gonna upgrade equipment or are better than most at culling. I've now got a pixel shift capable camera, and one RAW runs over 100MB.

I regularly move photos off the boot SSD to either an attached Thunderbolt SSD (didn't get it for that; was leftover equipment from another computer setup) or a HHD via USB 3. Because of how Lr handles the previews I don't really notice much diff in accessing images as long as the previews are up to date, since they are in the place that gives the fastest access: the boot drive.

Thanks for your input! I have a Nikon D800 (so big RAW files) and all images combined currently sum up to 275Gb. Timelapse sequence files are at 200Gb. So in total around 500Gb. I'm very strict in which images I keep so I expect that an external 3.0Tb in Raid(1), so 2x3Tb drives, will last me long enough until my next mac upgrade (currently on 2008 Mac Pro). I expect the memory usage to triple in about 5 years time = 1,5Tb on 3TB raid1 drives so I think I should be safe. My Lightroom catalog is about 15Gb on 11k images. There are things you can do to drastically reduce the catalogue size without LR becoming slow.

LR catalog + programs will go on the SSD (256Gb) and RAW images on USB3.0 3TB Raid 1 DAS. That's the plan!