Economical NAS & DAS with thunderbolt 2

[cmm]

100% true.

That being said, during all the time I have had a NAS with redundancy (over a decade), I have not needed any backups of the NASes nor have I lost any data. This includes the time when we were hit by Hurricane Sandy and lost power for over a week. Throughout all this time, redundancy to protect against failed drives have been sufficient.

Wow I hope the rest of your stuff electronic and otherwise had the same good fortune!

Two questions, do you use an UPS (if so, which one?) and which NAS do you have and what software version are you using? I'm curious since you said it defaults to afp.

I find storage to be certain important and most people throw up their hands in frustration and end up with 5 USB drives half of which are SSDs and are worthless.

I love helping out individually but I wonder how to scale it maybe write a quick we all to walk people thru options based on inputs. What do people here think? Is there any interest? Or am I solving a problem no one really cares about?

 

[pastrychef]

Wow I hope the rest of your stuff electronic and otherwise had the same good fortune!

Two questions, do you use an UPS (if so, which one?) and which NAS do you have and what software version are you using? I'm curious since you said it defaults to afp.

I find storage to be certain important and most people throw up their hands in frustration and end up with 5 USB drives half of which are SSDs and are worthless.

I love helping out individually but I wonder how to scale it maybe write a quick we all to walk people thru options based on inputs. What do people here think? Is there any interest? Or am I solving a problem no one really cares about?

Hardly good fortune. I've had my fair share of failed hard drives, but the redundancy was enough for me to replace the drives with new ones and not lose any data.

Yes, I use APC UPSes.

I currently have a Drobo S connected to an old Mac mini which handles the file sharing services. I also have an 8 drive unRAID array which I am transitioning to XPEnology (running SHR). I also had a very old 1U Snap Server (running RAID 5) which I retired a while back and I have a Drobo 5N which is sitting in a closet due to its 16TB limitation. The default behavior I spoke of regarding AFP is with the XPEnology.

In my opinion, having a bunch of individual external drives are the least ideal storage solution. The cable clutter and lack of redundancy make it messy and risky. Cheap enclosures can also add to instability and unreliability.

 

[cmm]

No, I meant with respect to the storm.

Okay, so I have much more experience with larger stuff, but I’d go with either:



1. ThunderBay 4 RAID mini enclosure for $429 and run in RAID 5

OR

2. Drobo 5D 5 bay storage for $673. NB This is Thunderbolt 1 but will be more than adequate for Lightroom thumbnails and pictures.



Note, both of these don’t use standard Linux software RAID and getting that isn’t possible since you don’t want to use your own hardware. As a result, I’d recommend 2 disk redundancy, if you can financially afford it. Both of the above lets you add drives later without issues AFAICT.


I recently picked up 5 1TB Crucial SSDs for $180 a piece refurbed from Crucial itself. These deals always come up. 2TB is too expensive.


I’d put as much as you really think is necessary on the SSD array and then the rest, to keep it simple and within the family, just get this, add your own drives option then get the 6TB SSDs for a grand total of ~$750 vs $878 if you let OWC add the drives. Use WD Red drives only!

http://eshop.macsales.com/shop/Thunderbolt/External-Drive/OWC/Elite-Dual-RAID NB This is Thunderbolt 1

As for RAID, in the ThunderBay 4 mini RAID, I’d recommend 1+0. For the Elite Dual RAID, I’d run RAID 0. Which requires backups:


As for backups, sign up for Amazon’s Cloud backup for $60 a yea) (or Backblaze if you want, but I like Amazon's cloud backup...you could get more advanced and rsync and/or snapshot to s3 but it isn't worth it . It is fast enough…if you have a good upload speed. And use your various physical USB3 external HDD’s as backups…you could get more elegant but then you are looking at $600+ for backups and you already have a bunch of external USB drives that will go to waste.

If you had $3200 just get this you’ll probably be happy, but you still need to buy the SSDs http://www.bhphotovideo.com/c/produ...t_i7_16g_us_tvs_871t_8_bay_thunderbolt_2.html

I hope that helps. If you skip SSDs all together I can make some better recommendations (better == better redundancy, lower overall price, etc) but you will be limited to gigabit ethernet which is about 120mbps (unless you use AC wifi which should be faster if the device and router is near your workstation) and quite frankly 120mbps is fine for photos. Another option is just buy this Thunderbolt 1 (not 2, but again, it’s fine for photos) dock for $175 (amazon) and then two 1tb SSDs and you are around the $500 mark. Seems to have some reliability issues but I didn’t look that closely: http://eshop.macsales.com/item/Highpoint Technologies/RS5212/

If you do that, then you can build a much larger, proper and 2 disk redunant (if one disk fails it can take awhile to rebuild and if a second disk fails during that time you lose everything, hence my love for 2 disk parity.

I hope that helps, @arbitrage. Sorry it took me so long to get back to you, I was super busy.

 

[dyn]

If you're gonna do a DAS you might as well do iSCSI rather than NFSv4, etc. But honestly, don't think too much about the DAS vs NAS terminology, DAS is a largely made up Mac term.

The only thing made up is the above as none of it is correct. DAS is Direct Attached Storage and revers to things like SAS/SCSI, SATA/eSATA, USB, Firewire and so on. NFS is used on NAS because it is a Network FileSystem. iSCSI can be both NAS and SAN but never a DAS. Thunderbolt is a bit of an odd ball though. It is used to run various protocols on top. One of those being SATA and another being TCP/IP (a very cheap way of getting a 10GbE network!). If the former is used then it is DAS but if the latter is used it is a NAS.

NAS, SAN and DAS are generic abbreviations for specific kinds of storage and has been invented by the storage world if we have to point out an inventor. There are big differences between them. If there isn't a network then it cannot be NAS nor SAN. If there is network it cannot be DAS. The difference between SAN and NAS is in how they provide the storage. A SAN provides it in a very similar way as a DAS (iSCSI is simply SCSI (used in DAS) over a network connection) but a NAS only provides a service (think network shares via AFP, SMB/CIFS or NFS). This is crucial when you need to have a device that provides storage over a network but makes it look like as if it is local.

When we look at the average consumer/professional simple DAS and NAS devices are just fine. No one there requires things like a SAN. If you want a proper NAS then stick to brands like Synology and Qnap.

In consumer electronics, a DAS is almost exclusively used in the context of Macs. Since this is a consumer website, my comments are (ready for it) *largely* appropriate.

In consumer electronics no one uses the term DAS. It's either an external disk or a NAS. DAS is generally used by the storage and server people. Blade servers have made the use of this terminology a necessity because it tells you where the storage is and therefore how you (the sysadmin) should handle it.

Most people think you can get an SSD and connect it via USB3 and you're set but you will lose speed. A lot of it.

Some people think that you lose lots of speed when you put an SSD on USB3 but this isn't true at all. It only applies to high end PCIe SSDs, ones that Apple is currently using in their Macs. This doesn't apply to SATA3 SSDs. When using those you won't find much difference between USB3 or Thunderbolt 2.

But the point is, apple wants to end afp and time machine is fraught with problems. Honestly, Apple is probably working on a replacement for Time Machine at this point that is built on NFSv4 rather than take out afp protocol and replace only that protocol with Time Machine as we know it.

Apple has defaulted to the use of SMB as of lately. El Capitan continues that path so it is far more likely that they'll make Time Machine work over SMB instead of NFS. I'm hoping they'll make it work over both though.

I've had my fair share of failed hard drives, but the redundancy was enough for me to replace the drives with new ones and not lose any data.

Be careful with that. When you do put in the new drive it has to rebuild and during that time the array is very fragile. The problem gets bigger when the drives in the enclosure age and with bigger capacity drives. There have quite some people where the entire array crashed during rebuild causing them to lose all of their data. That's why you should always have a backup. If you screw up the replacement of the faulty drive it can result into a full loss of data due to a broken array.

 

[cmm]

Theoretically, USB3 will do ~550-600 MB/s but in reality, the bus, the efficiency of the chipset, the overheard translation from one protocol to another, drivers, OS X peccadillos, Apple chipset, any other devices on the same bus etc.

True the OWC TB and OWC TB2 enclosures aren’t anywhere near as fast as the theoretical 10GB/s maximum of TB and 20GB/s maximum of TB2. Yet another reason to skip this "DAS" SSD crap (just because your boot drive has a PCIe SSD does not mean you need it for most any other applications other than OS, apps, and small config files for 99% of the word) and go with a proper WD RED ZFS (whether FreeNAS or CentOS using ZoL using 2 disk parity via RAID-Z2) array and connect it over NFSv4 and call it a day. Unless, of course you have $3500+ to burn on a proper TB2 device (excluding drives).

If you’re gonna do that, I like the Lenovo TS440, upgrade to at least 16gb--preferably 32GB--of ECC (very important!) RAM and get the hot swap bays for $20-30 on ebay for a 4 bay, and have a great system. Now the TS440 is $100 more in price than when I bought two, but with the hot swaps, you are looking at still under $500 sans drives. Spend a few hours configuring FreeNAS for your needs (plenty of info to get you thinking and guides online) and you’ll be ready to go. I don’t recommend using anything but Linux software RAID (unless you have enterprise needs that call for hardware RAID) so I inherently don’t trust drobo, Synology, QNAP, etc.



YMMV

 

[pastrychef]

Be careful with that. When you do put in the new drive it has to rebuild and during that time the array is very fragile. The problem gets bigger when the drives in the enclosure age and with bigger capacity drives. There have quite some people where the entire array crashed during rebuild causing them to lose all of their data. That's why you should always have a backup. If you screw up the replacement of the faulty drive it can result into a full loss of data due to a broken array.

I'm aware of that. It is for this reason that I like dual disk redundancy.

 

[pastrychef]

I recently picked up 5 1TB Crucial SSDs for $180 a piece refurbed from Crucial itself. These deals always come up. 2TB is too expensive.

At $180 each, that's $900 for a five drive RAID 5 array and you only end up with 4TB of storage with single drive redundancy.

With $900, you can easily get three 8TB drives and have change left over. You can use two of them for backups and still have more storage. However, your SSD array will unquestionably have far faster performance. So, it depends if the user is after performance or storage capacity.

 

[cmm]

If you read what I wrote to arbitrage and in general, SSD is rarely necessary, even for photos. I have needs for an SSD array, however. I also have a 6x6tb raidz2 array. I would NOT recommend the Seagate 8TB drives; stick to WD RED's which are maxed out at 6tb currently.

 

[pastrychef]

I have been running five of the 8TB drives in my Drobo S since about March or April and have had absolutely no problems. The only thing is that rebuilds take a long time.

 

[matreya]

If you read what I wrote to arbitrage and in general, SSD is rarely necessary, even for photos. I have needs for an SSD array, however. I also have a 6x6tb raidz2 array. I would NOT recommend the Seagate 8TB drives; stick to WD RED's which are maxed out at 6tb currently.

Personally, I wouldn't touch either the Seagate 8TB or WD Reds. The first is not designed for use in NAS and WD Reds are only designed for light use. The best drives bar none are the HGST Deskstar NAS series.

 

[pastrychef]

Drobo doesn't really seem to care what drives are used...

 

[chadi]

Hopefully this is on topic with the thread; and hopefully posting on a month old thread isn't going to get me flamed for dragging up old posts and I don't want to discuss the differences between NAS / DAS / iSCSI as that's been done already in here. But the QNAP TVS-871T that's been mentioned several times has me wondering if you can access the file system over Thunderbolt2 the same way you can over the network - all the QNAP units I have (which is about 10 different ones over the past 6 or 7 years ranging from 2 bay to 8 bay) have no direct connect option; and of course if you are using iSCSI you are formatting a new file system which is only accessible via iSCSI. It seems to be that the marketing is saying you can plug up to two machines into it with TB2 ports and access everything you could over the network; and it's even hot pluggable. If anyone has one and can comment that would be great.

To the OP, presuming you haven't already done something, after reading through the whole thread I'm not 100% sure why you want to direct connect? You aren't doing video editing just photography - I do not believe you will be unhappy with the performance if you put the catalog on the internal SSD on your Mac and store your photos on the Gbe network share from a 4 or 5 drive QNAP (I'd go 5 drive so you can do RAID6 with better capacity; 4 drives is a bit of a waste on space - and I suffered a 2 drive failure on RAID5 earlier this year on NAS rated drives and have since gone over to RAID6 as that was a big waste of time restoring data.

 

[calaverasgrande]

No mention of the OWC TB units?
Their prices on their mere TB units have gotten better since their TB2 units debuted.
Also, from time to time you can find the Drobo units for good prices. While Drobos are not the king of performance, they are stable and mature as a product. And are easier to use than some of the more leet-geek offerings.
I'd warn against the drobomini though. large capacity 2.5" drives are hard to come by. Doesn't seem to be much of a market for 2TB+ laptop drives.

 

[FireWire2]

Why don't you look at the thunderbolt 2 solution:
http://www.newegg.com/Product/Produ...96&cm_re=thunderbolt_2-_-16-151-148-_-Product
http://www.datoptic.com/ec/thunderbolt-2-eight-sata-sas-6gb-s-raid5-6-quite-tower.html

It gives you the FASTEST transfer, can make a share out if it via your Mac, so it will serve as DAS and NAS in the same box

Both of them offer online expansion, raid level mirgrate, Matter of fact they are almost identical in performance because I believe they use the same raid engine

But there is ONE major different and it's huge:

DATOptic is upgrade-able to thunderbolt 3, where none of thunderbolt , thunderbolt 2 from other vendor offer the same
[doublepost=1463090121][/doublepost]Addition to it
Thunderbolt 3 is right around the corner, why invest to a potential antiquate item