RAID Cards, Configurations, and the Mac Pro

[nanofrog]

Yeah, again this thread has been extremely informative and I hope other n00bs like myself find all this info useful in the future too! After getting everything up and running following last night's issue I have been testing the heck out of this thing.

After removing the jumpers I haven't found any issues yet. I've been pulling out random drives, 2 at a time, rebuilding, pulling out 3, watching the array disappear and seeing what the software does as you suggested. So far it seems very easy to deal with provided you know which drives fail. MY ADVICE to anyone doing this for the first time, do what I did and label the drive bays with the serial numbers of your hard drives (see previous pics). It has saved MUCH confusion already!!!!!

Anyway, The system still responds quite well with only 3 of the 5 drives and rebuild times of the other 2 seem like they are taking 3-4 hours in the event of dual drive failure. Not sure what was going on with those jumpers being on there in the first place but I'm glad you think that may have been THE issue.

Oh and... I see how this is MUCH easier than when critical data is on the drives! And more importantly I definitely see the need for a functioning backup system!!!!!

Isn't learning fun?

See? There actually is a method to my madness.

On a totally unrelated note you mentioned something about the Areca cards having partition table backup. Why is this a bonus? Oh and after all of this setup that I am doing, what is going to happen to my data in the event that my card fails? Does that mean that I will have to reinitialize them and I'd lose the data or if you put in the same controller card and you connect things the same does it recognize the array? I am assuming that a card death = array death since the setup of the array is going to be stored on the controller?

The PT table backup is an NVRAM solution to the write hole problem in RAID 5 & 6. It can allow you to recover the array, when it wouldn't be possible on other cards that don't have this feature. It's one of the reasons the card costs what it does, compared to say a Highpoint. (Areca designs and builds some of thier products, particularly the SAS cards they offer).

There's hidden commands for the Areca. If you ever have a problem, their tech support will walk you through it. Surprisingly easy. (I'd have to go back and look up the command, as it's slipped my mind ATM).

In the case of a card failure, you just replace the card. The drive DATA would still be intact. The primary PT data is on the drives, and in the case of the ARC-1231ML, an identical copy is kept on the card as well. You just install the new card, and attach the drives. NO REINITIALIZATION REQUIRED.

One of the nicest cards to have in the event of a failure. Some may not like the web browser interface to control the card, but I don't mind it. It works well, and isn't as daunting for less experienced users as the CLI (Command Line Interface) method. But I have to admit, Atto Technology has an even nicer interface. But their cards goes for more $$$.

However, you may get some corruption if the card was attempting a write when it died (limited to the specific file(s) being written at the time of the failure). In an enterprise environment, a spare card(s) are kept for just such an event. Along with drives, and even enough parts to build an entire system or two.

If you want to know what's going on, particularly with a failure, CHECK THE LOG FILES. These can save you all kinds of aggravation, including blowing the data, when it wasn't necessary. (It can show things like the drive order during a failure).

Though their native language isn't English, they do a fair job with the User's Manual. Read thoroughly, if you haven't already. It can answer a lot of questions specific to the card. Certainly can't hurt, and would really be a good thing to do. You might even pick up a few general tidbits as well.

 

[scaley]

Isn't learning fun?

See? There actually is a method to my madness.

The PT table backup is an NVRAM solution to the write hole problem in RAID 5 & 6. It can allow you to recover the array, when it wouldn't be possible on other cards that don't have this feature. It's one of the reasons the card costs what it does, compared to say a Highpoint. (Areca designs and builds some of thier products, particularly the SAS cards they offer).

There's hidden commands for the Areca. If you ever have a problem, their tech support will walk you through it. Surprisingly easy. (I'd have to go back and look up the command, as it's slipped my mind ATM).

In the case of a card failure, you just replace the card. The drive DATA would still be intact. The primary PT data is on the drives, and in the case of the ARC-1231ML, an identical copy is kept on the card as well. You just install the new card, and attach the drives. NO REINITIALIZATION REQUIRED.

One of the nicest cards to have in the event of a failure. Some may not like the web browser interface to control the card, but I don't mind it. It works well, and isn't as daunting for less experienced users as the CLI (Command Line Interface) method. But I have to admit, Atto Technology has an even nicer interface. But their cards goes for more $$$.

However, you may get some corruption if the card was attempting a write when it died (limited to the specific file(s) being written at the time of the failure). In an enterprise environment, a spare card(s) are kept for just such an event. Along with drives, and even enough parts to build an entire system or two.

If you want to know what's going on, particularly with a failure, CHECK THE LOG FILES. These can save you all kinds of aggravation, including blowing the data, when it wasn't necessary. (It can show things like the drive order during a failure).

Though their native language isn't English, they do a fair job with the User's Manual. Read thoroughly, if you haven't already. It can answer a lot of questions specific to the card. Certainly can't hurt, and would really be a good thing to do. You might even pick up a few general tidbits as well.

Thanks, for all of that information and I now consequently have an Areca on order as a replacement for this highpoint card. I've been copying to and from this array with the highpoint card and it has caused a LARGE number of lockups. I have verified the Array numerous times and it says it good but copy operations seem to lock my machine up now and I'm getting pinwheels of death. So I'm not exactly sure what is going on there but I had planned on switching for the Areca ARC-1680ix-8 anyway and so with everything that has been happening of late and the praise of Areca cards on here I think I will be happier. Other things were strange with that highpoint card in that the web GUI controller software was constantly failing on rebuilds and on initialization so I had to keep restarting it. Flashed the bios numerous times and updated but yeah, the behavior just got weirder the more I stress tested it. I also verified the hard drives and they are fine.

 

[nanofrog]

Thanks, for all of that information and I now consequently have an Areca on order as a replacement for this highpoint card. I've been copying to and from this array with the highpoint card and it has caused a LARGE number of lockups. I have verified the Array numerous times and it says it good but copy operations seem to lock my machine up now and I'm getting pinwheels of death. So I'm not exactly sure what is going on there but I had planned on switching for the Areca ARC-1680ix-8 anyway and so with everything that has been happening of late and the praise of Areca cards on here I think I will be happier. Other things were strange with that highpoint card in that the web GUI controller software was constantly failing on rebuilds and on initialization so I had to keep restarting it. Flashed the bios numerous times and updated but yeah, the behavior just got weirder the more I stress tested it. I also verified the hard drives and they are fine.

If you go with the ARC-1680ix8, remember Areca's SAS controllers are PICKY with SATA drives. It would require the use of enterprise models. (IIRC, I didn't see the consumer versions "passing" on the HDD Compatibility List). The SATA based cards are far more forgiving.

Also keep in mind, that even though a drive might verify properly when used as a single drive, RAID changes things. Particularly in how errors are handled. The timouts are different. It's part of the firmware, and is one of the primary differences between consumer and enterprise models. TLER values if you're using WD drives, and ERC with a Seagate.

 

[ncc1701d]

I'd skip Apple's card. I don't think you'd get enough of a performance boost to warrant it at all. RAID 0 is simple, and won't place much of a load on the CPU(s) and memory resources. It's cache will help, but I can't see it being worth the asking price. (Not justifiably at any rate). You'd likely be able to get just as much of a boost by carefully selecting your drives.

I only mentioned 3rd party, as that would be the way to go for better performance than the Apple RAID Pro card can provide. Additional drives (more than 4 ) would be needed.

Sorry to derail, just wondering, apart from the small speed increase form the apple raid card (for raid 0 only), is there any other benefit in having the raid card? (Does it give you more space per drive in the array?)

If I decide to just get 4 x 1TB drives in no raid configuration (2009 Quad), will each drive appear as a separate drive on my desktop?

For large music / movie storage, basic video editing, is there a web site or any suggestions on how best to "select drives"? Thanks!!!

 

[funkahdafi]

Sorry to derail, just wondering, apart from the small speed increase form the apple raid card (for raid 0 only), is there any other benefit in having the raid card? (Does it give you more space per drive in the array?)

If I decide to just get 4 x 1TB drives in no raid configuration (2009 Quad), will each drive appear as a separate drive on my desktop?

Yes, they will show up as four separate drives. But you can use disk util to setup a raid-0 array or JBOD. Then you will see a large 4 TB volume.

 

[nanofrog]

Sorry to derail, just wondering, apart from the small speed increase form the apple raid card (for raid 0 only), is there any other benefit in having the raid card? (Does it give you more space per drive in the array?)

NO, you won't get any additional capacity vs. a software implemented array. It's a function of the array configuration.

If I decide to just get 4 x 1TB drives in no raid configuration (2009 Quad), will each drive appear as a separate drive on my desktop?

YES, whether attached to a RAID controller or not. If you go with a hardware controller, the drives will still be listed individually after you create the array (array label followed by drives used), as it helps in diagnosing which drive is having problems, when they occur.

If you go software RAID, it will show up as a single volume (drive letter) after you set up the array.

For large music / movie storage, basic video editing, is there a web site or any suggestions on how best to "select drives"? Thanks!!!

I'm not aware of one, but it may exist.

I would just base it on need. Get an idea of the actual capacity you want, and go from there, if you're going to use RAID. You can make a calculation of the usable capacity from this. (Array type, # of drives, and individual capacities of the drives). Wiki's RAID page can help.

 

[scaley]

I would just base it on need. Get an idea of the actual capacity you want, and go from there, if you're going to use RAID. You can make a calculation of the usable capacity from this. (Array type, # of drives, and individual capacities of the drives). Wiki's RAID page can help.

This may also Help: RAID Calculator

 

[dagomike]

setup question

Any benchmarks out there comparing the software RAID 5 with hardware? Planning to do 3x 1 TB ES.2 drives in a new PM with a 1 TB boot/app drive.

Also will be using ExpressSAS R380 for an external eSATA box + LTO4 drive. Might not have the PCI bandwidth, or even if these two cards fit?

 

[nanofrog]

This may also Help: RAID Calculator

Cheater!

Any benchmarks out there comparing the software RAID 5 with hardware? Planning to do 3x 1 TB ES.2 drives in a new PM with a 1 TB boot/app drive.

Also will be using ExpressSAS R380 for an external eSATA box + LTO4 drive. Might not have the PCI bandwidth, or even if these two cards fit?

Check this page (search results). PM may slow you down, as it only uses a single port to transmit data. If so, it won't be by much, as 3 drives shouldn't tax it too hard.

I'm not sure what you mean about the PCIe Bandwidth issue. Could you explain?

 

[lieiti]

Also keep in mind, that even though a drive might verify properly when used as a single drive, RAID changes things. Particularly in how errors are handled. The timouts are different. It's part of the firmware, and is one of the primary differences between consumer and enterprise models. TLER values if you're using WD drives, and ERC with a Seagate.

First off, GREAT thread going here. A wealth of detailed information.

I just want to jump in here with a theory regarding SCALEY's Raid issues and what NANOFROG commented on regarding Hdd timeout values.

SCALEY, you previously stated that you were using "5, 1 Tb Seagate Barracudas (all ES.2 but 2 that will be replaced soon)", which I took to mean that 2 of your drives are not ES.2 enterprise drives. Is that correct?

If so, are they consumer drives? According to NANOFROG's provided link, desktop drives and enterprise drives come with different timeout values by default, so is it possible that those 2 drives are set at the default 0/0 sec timeout as opposed to the 7/7 sec recommended for RAID? Could that be the reason for the lockups your experiencing? Not sure if it's a good point or not, but just wanted to throw it in. Any comment NANOFROG?

Btw SCALEY, did you ever replace those drives or the 4320? Any updates on your RAID status?

On a personal note, I ordered a Hp 4320 card just before I read this thread, and am looking to do the same system. I bought a Mac Pro dual 2.8 (2008), and also ordered 4 WD1001FALS Caviar Black drives and a MaxConnect cable. Question for NANOFROG: though these drives are not on the HIGHPOINT compatibility list, do you think they will work? I'm guessing I should change the TLER values. And of course, pray.

Also, being that my OS drive is brand new, so I'm thinking about maybe only upgrading to 10.5.5 as opposed to 10.5.6. Trying to avoid the issues SCALEY is having, though it might be tough. Thoughts?

 

[nanofrog]

If so, are they consumer drives? According to NANOFROG's provided link, desktop drives and enterprise drives come with different timeout values by default, so is it possible that those 2 drives are set at the default 0/0 sec timeout as opposed to the 7/7 sec recommended for RAID? Could that be the reason for the lockups your experiencing? Not sure if it's a good point or not, but just wanted to throw it in. Any comment NANOFROG?

Very possible, if it's a combination of consumer and enterprise units.
If enterprise only, check the firmware revisions, as it may be different. Some cards REQUIRE

Btw SCALEY, did you ever replace those drives or the 4320? Any updates on your RAID status?

On a personal note, I ordered a Hp 4320 card just before I read this thread, and am looking to do the same system. I bought a Mac Pro dual 2.8 (2008), and also ordered 4 WD1001FALS Caviar Black drives and a MaxConnect cable. Question for NANOFROG: though these drives are not on the HIGHPOINT compatibility list, do you think they will work? I'm guessing I should change the TLER values. And of course, pray.

Also, being that my OS drive is brand new, so I'm thinking about maybe only upgrading to 10.5.5 as opposed to 10.5.6. Trying to avoid the issues SCALEY is having, though it might be tough. Thoughts?[/QUOTE]

 

[nanofrog]

If so, are they consumer drives? According to NANOFROG's provided link, desktop drives and enterprise drives come with different timeout values by default, so is it possible that those 2 drives are set at the default 0/0 sec timeout as opposed to the 7/7 sec recommended for RAID? Could that be the reason for the lockups your experiencing? Not sure if it's a good point or not, but just wanted to throw it in. Any comment NANOFROG?

Very possible, if it's a combination of consumer and enterprise units.
If enterprise only, check the firmware revisions, as it may be different. Some cards REQUIRE a specific version (earlier versions won't work). This is why it's best to check the card's Compatibility List first, to avoid issues, or at the very least, be aware of the ones you may encounter.

On a personal note, I ordered a Hp 4320 card just before I read this thread, and am looking to do the same system. I bought a Mac Pro dual 2.8 (2008), and also ordered 4 WD1001FALS Caviar Black drives and a MaxConnect cable. Question for NANOFROG: though these drives are not on the HIGHPOINT compatibility list, do you think they will work? I'm guessing I should change the TLER values. And of course, pray.

If they don't, you CAN change the TLER values , but you'd almost have to use a PC w/ floppy to do it. (Rather difficult on the Mac). There's other ways to do it, but it takes more effort (depending on what you have access to).

Also, being that my OS drive is brand new, so I'm thinking about maybe only upgrading to 10.5.5 as opposed to 10.5.6. Trying to avoid the issues SCALEY is having, though it might be tough. Thoughts?

It would be worth a try at least.

 

[scaley]

First off, GREAT thread going here. A wealth of detailed information.

I just want to jump in here with a theory regarding SCALEY's Raid issues and what NANOFROG commented on regarding Hdd timeout values.

SCALEY, you previously stated that you were using "5, 1 Tb Seagate Barracudas (all ES.2 but 2 that will be replaced soon)", which I took to mean that 2 of your drives are not ES.2 enterprise drives. Is that correct?

If so, are they consumer drives? According to NANOFROG's provided link, desktop drives and enterprise drives come with different timeout values by default, so is it possible that those 2 drives are set at the default 0/0 sec timeout as opposed to the 7/7 sec recommended for RAID? Could that be the reason for the lockups your experiencing? Not sure if it's a good point or not, but just wanted to throw it in. Any comment NANOFROG?

Btw SCALEY, did you ever replace those drives or the 4320? Any updates on your RAID status?

On a personal note, I ordered a Hp 4320 card just before I read this thread, and am looking to do the same system. I bought a Mac Pro dual 2.8 (2008), and also ordered 4 WD1001FALS Caviar Black drives and a MaxConnect cable. Question for NANOFROG: though these drives are not on the HIGHPOINT compatibility list, do you think they will work? I'm guessing I should change the TLER values. And of course, pray.

Also, being that my OS drive is brand new, so I'm thinking about maybe only upgrading to 10.5.5 as opposed to 10.5.6. Trying to avoid the issues SCALEY is having, though it might be tough. Thoughts?

Hey sorry about the delay, been away. Yeah so I did actually end up replacing the drives with 2 more ES.2's and the timeout issues I had actually were solved by the limiting bandwidth jumpers that were set on a few of them. Removing the jumpers solved those issues but I was still not happy with how quirky the 4320 was. So I had a 1680ix-8 on order but then decided to cancel that and go with the Arc-1231ml based on the praise it was receiving and the fact that I have Sata drives. So, now I'm just waiting on that to arrive and I currently have a hack job of backups and hard drives strung together on software mirroring until it shows up. But it should be here in the next few days and then good things will happen!

 

[joaoferro37]

Hi,

I will order my 2009 Mac Pro this week and was wondering to order the RAID card with it or not. If I am only doing RAID-0 or RAID-10, will the card give me any performance gains over Mac OS X's software RAID?

I am getting different anwers for this so I figured I'd ask in this thread as you guys seem to really know what you are talking about.

Thanks, and sorry for hijacking the thread.

you have one option,
Get the TimeLine Digital Pro Drive
http://www.timelinedigitalinc.com/stardom-drive-p-561.html
Pro Drive will take two 2.5" drives in one Mac Pro internal slot.
Each slot will give you around 130 MB/s (hardware RAID 0)
Put your OS drive at the optical bay, and install total of 4 Pro Drives.
You will get 4 RAID 0 volume, then use OS RAID 0.
You should get over 400MB/s

 

[joaoferro37]

Scaley,

I've had very good experience with the Areca RAID controller. I'm using the ARC-1212X controller, with 4x 1TB drives in a CineRAID 4 bay tower. I'm getting over 310MB/s read and write speeds in RAID 0 using the AJA Kona test. Even with my storage capacity filled at about 30% full.

Also, if your still looking for the MiniSAS cables, check out http://www.tekramonline.com/Accessories_Cables2.html, they have the external MiniSAS cables for $59

There is a special on their authorize distributor website
http://www.timelinedigitalinc.com/cineraid-c-93.html?osCsid=5971c2fb0aee2d8bdcaf3dcfb152523e
8TB with Areca ARC-1221X for $2979.
I stopped by CineRAID's booth and foudn out a lot of exhibitors such as, AJA, Blackmagic Design, Gfass Valley, SmallTree, even some rackmount server manufacture are using their controller cards.

 

[joaoferro37]

Thereis another option,
Using Areca's controller card and hooked up with ProAVIO's EDITBOX 8MS.
I have few clients are using the ARC-1221X
http://www.span.com/catalog/product_info.php?products_id=22324&source=gbaseus&currency=USD
From B and H
http://www.bhphotovideo.com/c/product/575945-REG/PROAVIO_EB8MS_8_0TB_8TB_editBOX_8MS_Array.html

 

[nanofrog]

Areca and Enhance both make really good gear, and are fairly well known these days. Not the cheapest available, but it's stable, and performance is usually quite good.

 

[joaoferro37]

Areca and Enhance both make really good gear, and are fairly well known these days. Not the cheapest available, but it's stable, and performance is usually quite good.

I do agree and I was really impressed at the NAB show.
Of course, there are more selections out there, such as G-Tech, Drobo and more but having some degree of DIY is always benificial for end users.

 

[Rx7MR2]

I was led by people from the Eyeon booth to the CineRAID booth, because they were very impressed with their 12 Bay expander solution, which they were using at the show for their live demonstration.

Areca/CineRAID was running their new 12 bay expander tower at the booth. Very impressive and looked great. I think it was the best looking out of the G-tech, Enhance or Caldigit solutions. I felt CineRAID had the best products at NAB this year. Everyone else was showing alot of the same old products of last year.

They also had a sweet 16 bay rackmount and told me that a 24 bay rackmount will be available soon. But, that's way to much for me.

 

[nanofrog]

I do agree and I was really impressed at the NAB show.
Of course, there are more selections out there, such as G-Tech, Drobo and more but having some degree of DIY is always benificial for end users.

There's a few decent products out. Generally speaking, I like DIY for home users/SOHO for cost reasons. If a purchase is possible/warranted, I like a product that's a little simpler, as there's less to go wrong.

I really wish I could have gone to NAB, but it wasn't possible. No budget for it, and I wasn't willing to pay for it myself.

Areca/CineRAID was running their new 12 bay expander tower at the booth. Very impressive and looked great. I think it was the best looking out of the G-tech, Enhance or Caldigit solutions. I felt CineRAID had the best products at NAB this year. Everyone else was showing alot of the same old products of last year.

I'll take another look at their site.

I keep thinking Areca might be up to something for faster SATA cards. The 12xx SATA (IOP 341) units have been around since 2006, and the 6.0Gbps components are available. (No 6.0Gbps drives yet, but why not; it's an excuse...)

Did you pick up on anything?

They also had a sweet 16 bay rackmount and told me that a 24 bay rackmount will be available soon. But, that's way to much for me.

Last I checked, a decent 24 bay rackmount went for ~$1100 IIRC. By no means inexpensive. But at least it's cheaper than 3x 8 bay units.

 

[Rx7MR2]

Nano,

Areca mentioned that their 6G cards will be out by Q4. They too mentioned that there was no reason for a 6G RAID controller if no 6G drives are available. Although they did confirm their beta testing 6G drives.

The only other new card they had at the show was their ARC-1300 series, which are non-RAID HBA's. They showed me the little 4 bay RAID box (ARC-5020) connected to the new Areca ARC-1300-4e (eSATA SAS HBA) putting out over 230MB/s using the AJA test.

 

[nanofrog]

Nano,

Areca mentioned that their 6G cards will be out by Q4. They too mentioned that there was no reason for a 6G RAID controller if no 6G drives are available. Although they did confirm their beta testing 6G drives.

The only other new card they had at the show was their ARC-1300 series, which are non-RAID HBA's. They showed me the little 4 bay RAID box (ARC-5020) connected to the new Areca ARC-1300-4e (eSATA SAS HBA) putting out over 230MB/s using the AJA test.

Testing makes sense, as they'd want to unveil around the same time the drives appear.

I've seen the 1300 cards on their website, but not gotten my hands on one.
How were the drives set up on the 5020?

 

[danielheldman]

NanoFrog,

I think you are one of the most knowledgeable users on this forum, I have a question for you.
If I use this CineRAID 12 bay with 24 SSD or 2.5" 7200 rpm drive will reach the speed limit of CineRAID 12 bay saying using Areca 1680 card?
I was at the NAB there was a product that can take 2 2.5" SSD or NB HDDs in one 3.5" HDD size enclosure. It has built-in RAID 0 or 1.
I talk to TimeLine Digital and they can build me one and I had a long discussion with them over the phone. They said, building a RAID 05 or 50 I cannot remember but would that be a safe setup.

The speed requirement is at least 1GB/s

Thank you for your reply.

 

[nanofrog]

NanoFrog,

I think you are one of the most knowledgeable users on this forum, I have a question for you.
If I use this CineRAID 12 bay with 24 SSD or 2.5" 7200 rpm drive will reach the speed limit of CineRAID 12 bay saying using Areca 1680 card?
I was at the NAB there was a product that can take 2 2.5" SSD or NB HDDs in one 3.5" HDD size enclosure. It has built-in RAID 0 or 1.
I talk to TimeLine Digital and they can build me one and I had a long discussion with them over the phone. They said, building a RAID 05 or 50 I cannot remember but would that be a safe setup.

The speed requirement is at least 1GB/s

Thank you for your reply.

Could you provide link for the CineRAID 12, and particularly the 2*2.5" to 1*3.5" adapters?

In general, the ARC-1680's can handle PM chips from what I've seen. The chip's P/N might make a difference, and would be advisable to do a search first, just to be sure.

Did the display use the Areca 1680 SAS cards with this enclosure, adapter, or even both? Or was it another card?

My biggest concern though, is the issues the SAS RAID adapters tend to have with consumer SATA drives. The array ends up unstable, which is useless to you.

Areca's tested SSD's with the ARC-1231ML, but nothing's been listed of any testing on the ARC-1680 series AFAIK. That or I didn't see it earlier today. It might be OK with a stripe or mirror, but I'm nervous of it, given what I've seen (ARC-1680ix12 in particular), behave when attached to consumer units. The only exception I recall, were WD's GP models, and only after the TLER values where changed. Keep in mind, it was while testing RAID 5 and 6.

Again, a search might give some results of someone who's tried SSD's on one the 1680 series. I can't say from experience, but I'd be worried until I got definitive answers.

Sorry I can't offer more.

 

[VirtualRain]

I believe this is the new enclosure he's talking about...
http://oc.adata.com.tw/1_product_detail.asp?pid=ASX1SUMAL-2

Unfortunately, there's no specs on the controller used.