Best (and cheapest) solution for a large amount of storage (10-20tb)

[zikol]

I'm looking for a good way to store 10 to 20 TBs of data.

It needs to be readily accessible, but it doesn't need to be extremely fast. A normal usb 2 connection is fine, as well as firewire or esata or ethernet. I do transfer large files, but there is no urgent need for them to be lightening quick.

As it stands now, I have several external drives just hooked up to my computer. I'd like to avoid continuing that method. I want only one volume to show up on my mac. It doesn't have to be through raid striping, though I don't really know of another way; maybe just a container volume that has all the drives in it?

It'd be sort of nice if I could access these files from several computers over my local network, though not absolutely necessary.

It needs to be fairly inexpensive. I need at least two of these systems to ensure I don't lose the data, so cost is doubly a factor. Part of the reason I've got so many singular external drives is because they're cheap (like $70 a TB), and I'd like to stay around that general range. So total cost should not exceed $100 per TB at most.

Finally, I'd like to be able to turn it off by simply ejecting the volume and hitting the power switch. No shutting down, or unplugging, or whatever else.

I've been looking at the Drobopro 8 bay storage array, and it seems near perfect, but it's just so expensive (~$2200 for Drobo + 16TB of HDDs)(~$137 per TB). Other thoughts include the Stardom Sohotank ST8, but from what I can tell about it is that it is a simple enclosure.

Any advise or ideas would be welcome.
Thanks in advance.

 

[pastrychef]

In my opinion, all the Drobos are too expensive except for the 4 bay model. You can do what I did and get two 4 bay models. I shopped around and managed to get them for about $320 each. Then, toss eight 2TB drives in them. 2TB drives can often be found for about $100 each now.

$320 Drobo
$320 Drobo
$800 eight 2TB drives
-------
$1440 16TB @ less than $100 per TB (however, 4TB will be used for redundancy)

You can also try to build your own NAS. There are many variables to this option...

 

[reebzor]

Build a server.

I built an 8bay server (enough for 16tb) for around $700 without disks. I used a Norco case (they have lots of options including up to 24 bay servers) then you'll need to buy a raid card. I have a Highpoint RocketRAID 2720 and even though I've heard some bad stuff about them, I've never had any problems. I bought a cheap motherboard, i3 processor, and a 2gb stick of ram to throw in there and it was perfect. I've used different NAS products but nothing seems better than the home server option.

You can certainly build one a lot cheaper than mine, especially if you already have an old PC lying around. I just chose to buy new hardware because I didn't want to have to update eventually.

 

[zikol]

I found some servers made by Lime Tech, does anybody have any experience with these? and would they be what I was looking for?

 

[pastrychef]

I have a Lime Technology unRAID setup. It requires a bit of work but it works well.

 

[AnalyzeThis]

Build a server.

I built an 8bay server (enough for 16tb) for around $700 without disks. I used a Norco case (they have lots of options including up to 24 bay servers) then you'll need to buy a raid card. I have a Highpoint RocketRAID 2720 and even though I've heard some bad stuff about them, I've never had any problems. I bought a cheap motherboard, i3 processor, and a 2gb stick of ram to throw in there and it was perfect. I've used different NAS products but nothing seems better than the home server option.

You can certainly build one a lot cheaper than mine, especially if you already have an old PC lying around. I just chose to buy new hardware because I didn't want to have to update eventually.

+2c. I did the same, only used "Windows 7" + set sharing security the same way as WHS does on RAID50 volume(s) (8x2TB - 2x2TB = 12TB). It was way better investment then any COTS NAS enclosures. I used Antec case for $59 (9x5" bays[needs a little metal bending, but gorgeous air flow case]) + 2xSuperMicro hot-swap bays ($99: 5 disks, but only 4 are used by Seagate LP 2TB HDD). Total power consumption 80W (idle), whisper quiet (changed SM bays noisy fan with quiet ones). Finally, network is the bottleneck: Steady 99% 1Gb network utilization ~112MByte ps.

Another very economical option would be FreeNAS with softRAID5. It is ok on 1Gb (50-60MBps) or 100Mb networks, but chocked on mixed network (100Mb transfer rate < 1.8MBps [should be ~10MBps]).

 

[jav6454]

I'm looking for a good way to store 10 to 20 TBs of data.

It needs to be readily accessible, but it doesn't need to be extremely fast. A normal usb 2 connection is fine, as well as firewire or esata or ethernet. I do transfer large files, but there is no urgent need for them to be lightening quick.

As it stands now, I have several external drives just hooked up to my computer. I'd like to avoid continuing that method. I want only one volume to show up on my mac. It doesn't have to be through raid striping, though I don't really know of another way; maybe just a container volume that has all the drives in it?

It'd be sort of nice if I could access these files from several computers over my local network, though not absolutely necessary.

It needs to be fairly inexpensive. I need at least two of these systems to ensure I don't lose the data, so cost is doubly a factor. Part of the reason I've got so many singular external drives is because they're cheap (like $70 a TB), and I'd like to stay around that general range. So total cost should not exceed $100 per TB at most.

Finally, I'd like to be able to turn it off by simply ejecting the volume and hitting the power switch. No shutting down, or unplugging, or whatever else.

I've been looking at the Drobopro 8 bay storage array, and it seems near perfect, but it's just so expensive (~$2200 for Drobo + 16TB of HDDs)(~$137 per TB). Other thoughts include the Stardom Sohotank ST8, but from what I can tell about it is that it is a simple enclosure.

Any advise or ideas would be welcome.
Thanks in advance.

Look into 5 to 10 of these... Click Me and use Promotion code EMCZYNW48 to get each one for $89.99.

 

[reebzor]

Those Limetech servers are nothing more than a custom built PC. Anyone could do it themselves but probably save a couple hundred bucks.

Not saying you shouldn't buy one, but you have the option of customizing the parts a little more and potentially save some money.

 

[reebzor]

Another very economical option would be FreeNAS with softRAID5. It is ok on 1Gb (50-60MBps) or 100Mb networks, but chocked on mixed network (100Mb transfer rate < 1.8MBps [should be ~10MBps]).

My experience with FreeNAS wasn't the best, but I would still recommend it for the average user.

I wanted to go the hardware RAID route instead of software, so I bought a nice RAID card. The card supports expansion and RAID migration so I wanted to utilize that and migrate from 4tb RAID0 to 6TB RAID5. With FreeNAS I only had the option to format the drives UFS and I was unable to expand the partition over 2tb because of the 32bit instance of growfs. I then nixed FreeNAS in favor of Snow Leopard Server, which installed almost perfectly on the hardware I had. HFS+ in OS X gives me the option of resizing partitions on the fly, which is necessary for me.

Just make sure everything you want to do (or are going to want to do) are supported by the OS/Hardware that you choose. Backing up and formatting a 4TB array was not much fun for me so I doubt it would be any better for a 10-20TB array.

 

[AnalyzeThis]

My experience with FreeNAS wasn't the best, but I would still recommend it for the average user.

I wanted to go the hardware RAID route instead of software, so I bought a nice RAID card. The card supports expansion and RAID migration so I wanted to utilize that and migrate from 4tb RAID0 to 6TB RAID5. With FreeNAS I only had the option to format the drives UFS and I was unable to expand the partition over 2tb because of the 32bit instance of growfs. I then nixed FreeNAS in favor of Snow Leopard Server, which installed almost perfectly on the hardware I had. HFS+ in OS X gives me the option of resizing partitions on the fly, which is necessary for me.

Just make sure everything you want to do (or are going to want to do) are supported by the OS/Hardware that you choose. Backing up and formatting a 4TB array was not much fun for me so I doubt it would be any better for a 10-20TB array.

Well, I glad I nuked freeNAS. I also tried Windows 2008 Server - did not like it, because of some secondary drivers failed not work/load on it. Windows 7 was a surprisingly fast and stable. If HDD cost exceeds cost of where you put them - you are saving money. There is a few tricks to optimize W7 for network sharing. It is interesting, you were able to run Mac OSX Leopard on your setup, was it very difficult? I was contemplating gradual expansion/migration route, but because of I was not sure how W7 going to react to changes in volume size, I decided to just load it up and not to take any chances. >10TB is serious liability to maintain, mistakes are very costly (time and effort).

 

[zikol]

I have a Lime Technology unRAID setup. It requires a bit of work but it works well.

By a "bit of work", what exactly do you mean?

If I were to build a server basically like the lime tech, and then use their unRAID software, would it require a lot of tinkering, or just install the os and that'd be that?

 

[pastrychef]

By a "bit of work", what exactly do you mean?

If I were to build a server basically like the lime tech, and then use their unRAID software, would it require a lot of tinkering, or just install the os and that'd be that?

Some of my initial experiences with unRAID are documented at:

https://forums.macrumors.com/threads/842539/

I had to tinker a bit more than most because I wanted AFP. If you do not need it, it's a pretty easy system to set up.

Since I initially set it up, I've added an external 4 bay hard drive enclosure (connected via eSATA) and expanded my array to 8 drives and I've upgraded two or three small drives to larger capacity ones.

 

[Cynicalone]

I use the DroboPro, and also a second generation Drobo.

They are pricey but they work really well. And with the release of 3TB internal drives it is very easy process to expand the Pro from 12.5TB To 18.9TB.

Data Robotics is in the process of certifying the Western Digital 3TB Greens so when you are ready to expand the Drobo will be able to handle the larger drives.

IMO, the Drobo's are a great way to have mass storage without the complications of maintaining a large RAID array.

 

[reebzor]

Well, I glad I nuked freeNAS. I also tried Windows 2008 Server - did not like it, because of some secondary drivers failed not work/load on it. Windows 7 was a surprisingly fast and stable. If HDD cost exceeds cost of where you put them - you are saving money. There is a few tricks to optimize W7 for network sharing. It is interesting, you were able to run Mac OSX Leopard on your setup, was it very difficult? I was contemplating gradual expansion/migration route, but because of I was not sure how W7 going to react to changes in volume size, I decided to just load it up and not to take any chances. >10TB is serious liability to maintain, mistakes are very costly (time and effort).

OSX was an easy install. I used the iBoot + Multibeast method from tonymacx86. I purchased the hardware with the possibility of hackintosh in mind, although it wasnt my initial intent.

I also contemplated using Windows 7/WHS/2008R2/etc for the server, but I opted not to. I use windows servers all day at work (I am a Network Administrator) and since at home I only have Macs, I wanted to see if I could get OS X working first. Windows was an absolute last resort for me. Since you are on windows though, if you plan on doing an expansion it is super easy to expand your partition in Disk Management, assuming the array is not your boot partition, which it shouldn't be anyway.