Need advice on the right Raid configuration

[cmuriel]

Hello, I am about to install 4 Internal 2T HD's in my Mac Pro 2.26 GHz to be used as the home media server in my house. I have over 1.5T Video to store, with another 1T to 2T to burn, and would highly appreciate any advise on what would be the right RAID configuration to use, and if possible, instructions on how to set it up. My goal would be to have the highest possible storage capacity and speed. I have reviewed the 0, 5, and 10 options, as well as, the stripped and mirrored configurations but I am lost, although I am leaning towards RAID 0. Am I on the right track?
I have an external 2T WD Drive to back up through Time Machine, so I don't have to worry about erasing and formatting all disks.
Can anybody help me, please??

[alphaod]

Well for starters, RAID0 would definitely offer you the most speed and capacity with your available disks. Now understand RAID0 isn't really RAID at all since it does not offer redundancy; therefore if one drive fails, all data is lost.

RAID5 may offer a good performance and capacity compromise as RAID10 will half the capacity of the sum of your disk volumes. If you do choose to use RAID5, please buy a good hardware RAID controller (that means don't buy the Apple RAID card which is a POS). RAID5 would give you around 6TB of total space.

For your external, 2TB would not be even be close to enough since your combine capacity of 8TB (RAID0) or 6TB (RAID5) cannot fit on a 2TB external.

Finally RAID5 will require an PCIe controller; and it would need to be bootable which will cost $400+.

[nanofrog]

Originally Posted by cmuriel

Hello, I am about to install 4 Internal 2T HD's in my Mac Pro 2.26 GHz to be used as the home media server in my house. I have over 1.5T Video to store, with another 1T to 2T to burn, and would highly appreciate any advise on what would be the right RAID configuration to use, and if possible, instructions on how to set it up. My goal would be to have the highest possible storage capacity and speed. I have reviewed the 0, 5, and 10 options, as well as, the stripped and mirrored configurations but I am lost, although I am leaning towards RAID 0. Am I on the right track?
I have an external 2T WD Drive to back up through Time Machine, so I don't have to worry about erasing and formatting all disks.
Can anybody help me, please??

RAID splits into two basic categories:

• Software RAID (includes FakeRAID controllers) = driver based, and uses the system resources (CPU and memory) to do the calculations.

• Hardware RAID = it's own processor and cache, which performs all the work (calculations) thus removes the load from the system resources all together.

Software RAID is the least expensive, and OS X is only capable of 0/1/10.

0 offers the highest throughputs, and capacity, but no redundancy as alphaod indicated.

10 offers decent speed, but it's not the fastest, while given a high level (2 disk) of redundancy. There's a cost however, which is capacity. for 8TB of raw capacity, you'd only be able to access half (4TB). I presume this will be a problem in short order, as you'd almost be full once you get your existing data transferred to it.

Software RAID (or FakeRAID cards = eSATA with RAID functions in the drivers) will claim to offer level 5 with some, but it can't deal with the write hole issue (see RAID wiki). For this, you'd need a proper hardware card, which is on the expensive side, and it's worse in the '09 systems, as you'd need an adapter to use the HDD bays.

The least expensive card worth having (SAS, 4 port, EFI bootable), would be the ARC-1212 (~$340USD). Then add in $165USD for the adapter, available from MaxUpgrades (here = only supplier). Then consider that with such a card, you must use enterprise HDD's (SAS cards are picky, and you'd want enterprise anyway, given their higher reliability due to the fact they're built for such use). WD's RE4 or RE4-GP to get 2TB disks that would work ($300 - 400USD each, depending on which model). Generally speaking, you should consult the HDD Compatibility List to be sure the drives will work, otherwise you're a guinea pig (i.e drives may not be stable). Not all card makers have such lists, but it's a good idea to look first before ever buying anything, as it can save you tons of headaches (lost time + frustration from hell + returns ( = extra $$)).

RAID /= Backup (no matter the array level used), so you will need to have a methodology (FW via TimeMachine may be too small, as I did notice you mentioned it). An eSATA card + Port Multiplier enclosure might be worth considering for this. It's simple, and inexpensive compared to other alternatives.

[beaker7]

If you are just using it as a home media server, speed really should not be a concern. Even a single hard drive will more than saturate a gigabit ethernet connection, and even then, you're not going to need anywhere near that speed to stream even 1080p video.

What media serving software are you going to use? If you do not need everything to appear on one volume in this case there is little need for RAID.

If you need everything on one volume, either do JBOD or RAID 0 and back it all up to a Drobo.

If your media manager can deal with things on seperate volumes, leave the 4 drives each fly solo and back them all up to a Drobo.

[mikes63737]

Are the drives made by WD? Some consumer-class WD drives might not work in RAID.

Read this: http://wdc.custhelp.com/cgi-bin/wdc....p?p_faqid=1397

Originally Posted by Link

If you install and use a desktop edition hard drive connected to a RAID controller, the drive may not work correctly. This is caused by the normal error recovery procedure that a desktop edition hard drive uses.

A reviewer on Newegg confirmed this problem...

[nanofrog]

Originally Posted by mikes63737

Are the drives made by WD? Some consumer-class WD drives might not work in RAID.

Read this: http://wdc.custhelp.com/cgi-bin/wdc....p?p_faqid=1397

A reviewer on Newegg confirmed this problem...

This can be a major PITA, which is why you check the HDD Compatibility List (if it exists).

SATA controllers are a little easier than SAS (super picky about it), but it has to do with the recovery timings programmed into the drive firmware. Consumer drives use 0,0 (seconds; read, write respectively) <individual drive settings>, while enterprise units use 7,0 as the default values <RAID settings>.

The reason this exists is due to how recovery occurs (low level operation). For consumer models, it's expected the drive will be connected to the SATA ports on the main board, and handled by the OS. RAID cards OTOH, do it all themselves, and it's in sets, not single drives (save Pass Through mode = individual drives).

[Fomaphone]

i've got a question of my own... i'd like to start planning (saving) for an adequate RAID system for editing of 2k video (shot 4k) since i find myself doing more and more of it at home.

basically, i'd like to DIY something as close to the 8-bay CalDigit HD Pro raid setup as possible for much less money. i'd like the RAID to be completely external in order to use my internal 4 bays for my own work, media collection, and some backup storage of original media files. i have a march 09 octad 2.66 with 12gb RAM.

the external raid would be for renders, reading/writing while editing, and storage of transcoded media

i'd really appreciate any recommendations or advice you can offer.

edit: i like this setup, but it's from 2008. what do you think, nano?

http://lifezero.typepad.com/blog/200...-under-2k.html

[joaoferro37]

Check this blog.
http://silveradosys.blogspot.com/
"

We did some testing in the shop today with new firmware on the ATTO R380 SAS card and the Stardom SOHOTANK ST8-U5 running Hitachi 2.0 TB drives. The results were some of the best I've ever seen in an 8-drive RAID 5.

Using a 16.0 GB sized file in 2048x1556 10-bit RGB format, the drive pushed 705 MB/s writes and 775 MB/s reads.

Further, using a 16.0 GB sized file in DVCProHD 1080i60 format, the drive pushed over 822 MB/s read speeds.

"

[nanofrog]

Originally Posted by Fomaphone

i've got a question of my own... i'd like to start planning (saving) for an adequate RAID system for editing of 2k video (shot 4k) since i find myself doing more and more of it at home.

basically, i'd like to DIY something as close to the 8-bay CalDigit HD Pro raid setup as possible for much less money. i'd like the RAID to be completely external in order to use my internal 4 bays for my own work, media collection, and some backup storage of original media files. i have a march 09 octad 2.66 with 12gb RAM.

the external raid would be for renders, reading/writing while editing, and storage of transcoded media

i'd really appreciate any recommendations or advice you can offer.

edit: i like this setup, but it's from 2008. what do you think, nano?

http://lifezero.typepad.com/blog/200...-under-2k.html

The basic methodology of the link provided is good, but I wouldn't use the Highpoint 2322, as its a FakeRAID controller (no processor or cache).

You'd want a real hardware controller, such as the Highpoint 4322, and an 8 bay MiniSAS enclosure. You will need 2x cables (here - stick to the 1M length, or it will cause the array to be unstable), and of course drives.

For drives, select something off of this list. Of what's listed, I'd go with WD. It would be even better to run enterprise drives (there are some consumer models listed). The Caviar Black 1TB at a minimum (WD1001FALS).

This card is inexpensive for what it is, and is a much better card than either the 2322 or CalDigit by a long shot. It's even priced well, which is why I list it. Areca's offerings are better (they ODM'ed it for Highpoint), but also cost more (~$100USD <or more if you want additional ports> due to additional features).

[Cathode]

Originally Posted by cmuriel

I am leaning towards RAID 0. Am I on the right track?

If you really want the performance gain from it you should really invest in a hardware RAID controller, I found that many software RAID0 setups are just a plain waste.

Also: Backup, backup, backup!

[ncc1701d]

With the amount you're looking to store, you are doing a similar thing to what I am. I have 4x1TB in hardware Raid 0, all being backed up to a drobopro through time machine. I'm considering getting the WD RE4 2TB's to replace the 1TB's, but the situation is the same. Not cheap and probably not the fastest (for back up), but it works a treat.

from all accounts, the apple hardware raid is not very good. I haven't had any problems with it per se, but I'm not a professional. nanofrog may be better to point you to hardware raid that's better.

[joaoferro37]

One more thing, make sure ask your vendor to use the Enterprise Level HDD.

The non-recoverable rate on the Enterprise Level drive is 1 in 10^15

http://www.hitachigst.com/tech/techlib.nsf/techdocs/5EDA5472B7792D9D8625760A0003E88B/$file/UltraStarA7K2000_datasheet.pdf

VS

The non-enterprise level is 1 tin 10^5

http://www.hitachigst.com/tech/techlib.nsf/techdocs/6A7E7E6848832B7786257603007AAF5E/$file/DS7K2000_DS_final.pdf

This means if you use a 2TB, and your data read rate is accumulated to 12.5TB, you will have an un-recoverable error. In another word, if you put 2TB date in your 2TB drive and access to the 2TB data more than 6 times, you will lose your drive.

[cmuriel]

I very much appreciate all your advise. However, this is becoming too technical for my level, apologies. So, I guess that if I want to have maximum storage capacity and speed, I should go a Raid 0, even though I understand I would not have redundancy. That's clear. Now, Today I have around 1.8T in media files that I have backed up in my external 2T drive. So, no issue here so far, however, if I go over the capacity of my back up drive, can I just connect another one to increase capacity? Do I do it by configuring them in Raid 0 as well?. Finally, I realized I've probably made a beginners mistake since I bought a MacPro Raid Card already. Is there any use installing it? or should I just try to return it, unless you highly recommend that I go for another Raid configuration.
Again, my apologies for what probably are stupid questions.
Thanks

[cmuriel]

Btw, I am using Apple TV's on my TV's and I tunes through a ethernet home network.