RAID5 options for Mac pro

[nanofrog]

Is that all you got out of the link?

Read this:

http://www.miracleas.com/BAARF/RAID5_versus_RAID10.txt

S-

I have read it, and I have some issues with it. Not with what's there necessarily, but what's missing.

1. It doesn't cover the use of NVRAM solutions, or their affectiveness.

2. It's old. It's discussing IDE and SCSI. SATA and SAS though having thier basis on the tech discussed, are different. Commands have been added that's under the card's control, not the OS.

3. It makes no consideration of specific differences that make one over the other a more viable solution in certain situations. Namely limited drive locations, capacity requirements, or finances.

All of which seem to be of concern with posts that come up. Users can't afford the external enclosures necessary to implement a large type 10 array, or the drive count, even when there may be budget enough for a decent card.

Keep in mind, an 8 bay external pedestal enclosure is ~$600 - 650USD (up from what it was a year ago, $550 - 600USD). Given the lack of a rack, rackmount enclosures aren't an option either. Before drives!

It gets ugly for individuals, and since many of those I've responded to are working with video/graphics editing, their capacity needs are high. So a few disks isn't going to cut it.

In the end, budget is the biggest pitfall. Otherwise, if funds are effectively unlimited (ignore budgets), a type 10 is a viable solution. I'd even recommend going with it vs. a type 6, as the capacity can be afforded, resulting in the same level of redundancy without the write hole issue (UPS or NVRAM based card + battery or not). Performance would be the only reason to change that, and as the budget's unlimited, just add more members to the type 10.

But we live in the real world, and money matters.

 

[daffydowden]

ok, first of all thanks for all of your help everyone, this has been incredibly useful. I'm still doing my research, so if I may, i'll continue to post questions...

Taking into account all of your advice, I'm swerving away from the ARC 1212 (No nvram, not bootable?) and instead thinking about the Areca ARC 1680LP (I like the fact it has both an internal and external port, for expandability; 256mb NVRam; and allows for a battery module) Am I correct in thinking that this drive isn't bootable? Is this just for OSx and would a Bootcamp partition on the drive work, or am I confused about how boot camp works? (I've never set it up yet)

If I therefore need a separate boot drive, could I use time machine to back up this drive on the RAID array? Is that sensible solution?

Finally, thanks for telling me about the drive sledges nanofrog, but if I buy this card and leads, am I committed to buying SAS drives as well? I just want to consider all of my options here. Anyone UK based know if there's any retailers over here that sell the SZ-MACPRO09-MS06 sledges?

Cheers,
Daf

 

[nanofrog]

Taking into account all of your advice, I'm swerving away from the ARC 1212 (No nvram, not bootable?) and instead thinking about the Areca ARC 1680LP (I like the fact it has both an internal and external port, for expandability; 256mb NVRam; and allows for a battery module) Am I correct in thinking that this drive isn't bootable? Is this just for OSx and would a Bootcamp partition on the drive work, or am I confused about how boot camp works? (I've never set it up yet)

The ARC-1212 (and the ARC-1222 = 8 port internal), are:
1. SAS based
2. Will Boot OS X
3. Has an NVRAM solution
4. Can use a battery backup unit; that's an option, same as the 1680 series (BBU)

There's no external ports on either though, so the ARC-1680LP will be a better choice for that. But you can use an internal to external cable through an open PCI bracket (nothing in the slot, just use the "hole" to run the cable through). It's cheaper than the ARC-1680LP. Cable + say ARC-1222 is less expensive (if you don't fill all the PCIe slots). The ARC-1680LP has a faster processor on it (1200MHz vs. 800MHz), but that won't make a difference with 8 drives. The 1680 series also adds the ability to run SAS expanders in order to run up to 128 drives. That's out of the realm in which you'll be running (really expensive for home use).

Boot Camp can NOT run on a RAID card or under a software RAID under OS X. If RAID is used for OS X, then Windows has to have it's own drive or array. The reverse is also true.

So some specifics as to OS's and what you're doing with them will help. If you need the array for OS X (and not for Windows), then use a single separate drive for Windows is the easiest, and least expensive way to go. Again, the reverse is also true. Dual arrays would be possible as well, but verify with Areca that the card will be able to boot both in a MP (I never got a clear answer on that one, but it might).

If I therefore need a separate boot drive, could I use time machine to back up this drive on the RAID array? Is that sensible solution?

You could do that. But make sure you have a proper backup solution for the array, and it can't be on the same drives (otherwise if they fail, you're screwed - both the original and the backup are lost).

Best to have a separate backup solution that can handle both (as if the array goes out, you lose the OS backup). You're OK unless the OS drive goes too (it can actually happen, such as a PSU going out, and taking other parts of the system with it).

Finally, thanks for telling me about the drive sledges nanofrog, but if I buy this card and leads, am I committed to buying SAS drives as well? I just want to consider all of my options here. Anyone UK based know if there's any retailers over here that sell the SZ-MACPRO09-MS06 sledges?

Cheers,
Daf

No, SAS cards can run SATA drives. But you do have to be carefull with drive selection though. That's why you check the HDD Compatibility List on Areca's site. Personally, I'm using drives from the Western Digital RE3 line.

The sled kit you need has to come from Maxupgrades AFIAK. They own the IP on the designs. So send an email regarding international shipping.

 

[slughead]

In the end, budget is the biggest pitfall. Otherwise, if funds are effectively unlimited (ignore budgets), a type 10 is a viable solution. I'd even recommend going with it vs. a type 6, as the capacity can be afforded, resulting in the same level of redundancy without the write hole issue (UPS or NVRAM based card + battery or not). Performance would be the only reason to change that, and as the budget's unlimited, just add more members to the type 10.

But we live in the real world, and money matters.

Exactly. I have a 4 drive RAID 5 composed of 1TB drives and a cheap card. I *NEED* that extra terabyte. I know there's a slight chance of failure, but it's still extremely remote I'll ever have a problem.

I don't have the money to but a good RAID card at this time (I don't even work in the field anymore, I'm using this space for hobbies/recreation)...

I appreciate the warnings of course, but it's not a choice between perfect and imperfect, it's between doing slightly riskily and doing without.

 

[sidewinder]

Exactly. I have a 4 drive RAID 5 composed of 1TB drives and a cheap card. I *NEED* that extra terabyte. I know there's a slight chance of failure, but it's still extremely remote I'll ever have a problem.

I don't have the money to but a good RAID card at this time (I don't even work in the field anymore, I'm using this space for hobbies/recreation)...

I appreciate the warnings of course, but it's not a choice between perfect and imperfect, it's between doing slightly riskily and doing without.

How are you backing up this array? Do you have NVRAM and battery backup? Are you using Enterprise class drives? Do you have a spare drive ready and waiting?

You thinking that there is an "extremely remote" "slight chance of failure" and using a cheap RAID card is exactly why RAID 5 should be vigorously discouraged for those with limited budgets and experience.

The argument by nanofrog that "money matters" is an argument against using RAID 5, not one for it. You need good money to buy the right card, drives, and backup solution. Having a spare drive ready is MUCH more critical with a RAID 5 array than a RAID 10 array. The bigger drives get, the worse RAID 5 is.

Where nanofrog gets the idea that RAID 5 out performs RAID 10 is a mystery. Only in read intensive (90+ percent) environments is RAID 5 faster than RAID 10.

The bottom line is that RAID 5 is rarely a good idea these days. Money issues notwithstanding.

S-

 

[slughead]

How are you backing up this array? Do you have NVRAM and battery backup? Are you using Enterprise class drives? Do you have a spare drive ready and waiting?

I'm not; I have a UPS, no NVRAM; no; no. I realize why they are important, I just don't have the money.

You thinking that there is an "extremely remote" "slight chance of failure" and using a cheap RAID card is exactly why RAID 5 should be vigorously discouraged for those with limited budgets and experience.

I'm sorry, but would you like to back up your statements with numbers instead of alarmism? What are the chances? what are the conditions in which fault may occur?

I think you'll find that the chances are "slight", and "extremely remote" if you bothered to do some research beyond the theoretical.

Yes, 10 is obviously better. However, it's also way more expensive. A BMW may be better than a Malibu, but it's moot because I can't afford a BMW!

Keep in mind, I can't afford a RAID 10! I'd have to buy 50% more hard drives to get as much space on a 10 as I have now with a 5.

Plus, with my budget, if I didn't have a 4 drive RAID 5, I'd have a 3 drive RAID 0... are the odds worse than that??

In the real world, money matters. I can't afford a RAID 10. Therefore, you're not telling me to get a RAID 10, you're telling me not to have a RAID at all.

 

[sidewinder]

slughead,

Can you afford to lose all of your data on the RAID 5 array? If the answer is no, then you should not have any RAID array. Not until you have a reliable backup plan.

If you can't afford to lose your data, you are nuts for running a RAID 5 array on a cheap card with no battery backup of the write cache.

S-

 

[BHamilton]

I can tell you right now, last night (before I installed the battery, which came a day later than the card), my Mac abruptly shut down (blink, lights out). Total loss of the file I was working on. UPS doesn't mean anything if you computer shuts down hard like that. I would NEVER run the RAID card without the battery again.

 

[nanofrog]

I'm not; I have a UPS, no NVRAM; no; no. I realize why they are important, I just don't have the money.

With NO backup, you're playing Russian Roulette with your data. Think of it this way; the odds are reduced for any single instance that the round will fire, but if you do it long enough, it eventually goes off, and the moron holding it is seriously injured or dead.

Same thing with your data. RAIDs do fail. When is the real question. So if you don't have a backup, you're going to be toast at some point. The card you're using makes it worse as well, as now you have higher odds that a write hole corruption will occur without the NVRAM solution.

I know money matters, but there's limits as to how inexpensively you can go without just being stupid. This isn't to insult you, but to try to get you to recognize your risk is too high. Pull back to single disk mode + backup, or a smaller stripe set with a backup. You can do this with the equipment you currently have. I'd rather see you with a stripe set + backup than a type 5 without one. The stripe set in this case is actually safer. This is what sidewinder is also trying to tell you, as your case is really precarious.

Had you an NVRAM based card combined with a proper backup solution, you'd be OK with the risk level. But that's not the case.

I can tell you right now, last night (before I installed the battery, which came a day later than the card), my Mac abruptly shut down (blink, lights out). Total loss of the file I was working on. UPS doesn't mean anything if you computer shuts down hard like that. I would NEVER run the RAID card without the battery again.

Did you upgrade to 10.6.2?

There seems to be problems with it and RAID cards (it's failed with Areca's, so it would affect your card as well). OS updates can break RAIDs, so you have to be careful. When it happens, you have to either roll it back, or restore from backups (depending on the extent of the OS files, as a roll back may not catch the offending file/s and setting/s).

 

[joaoferro37]

I would recommend avoiding RAID 5. You would be much better off with a RAID 10 solution. RAID 5 has too many issues to be used by the typical user.

There is the write hole issue, performance during rebuild issue, slow write performance issue, and potential for total data loss during rebuild issue. Avoid RAID 5 if at all possible.

S-

Can you explain a little bit more.
I have clients using caldigit hdpro and had not be able to recover data at all.
Can a RAID 6 setup avoid the problem?

 

[nanofrog]

Can you explain a little bit more.
I have clients using caldigit hdpro and had not be able to recover data at all.
Can a RAID 6 setup avoid the problem?

CalDigit's products suck, as does their support IMO. Missing features (i.e. copies of the partition tables stored in the card's firmware) is one major example, and is highly useful with recovering data.

To get into the specifics though, we'd need the details as to what happened.

They'd be much better off with cards from other vendors IMO.

 

[joaoferro37]

CalDigit's products suck, as does their support IMO. Missing features (i.e. copies of the partition tables stored in the card's firmware) is one major example, and is highly useful with recovering data.

To get into the specifics though, we'd need the details as to what happened.

They'd be much better off with cards from other vendors IMO.

WHAT!!!! the partition tables stored in caldigit's RAID card!!!!
That means if the card died, notthing will get recovered.
Am I correct!!!! Maybe that is why a lot of my clients are complaining about the failure of caldigit's products.

 

[nanofrog]

WHAT!!!! the partition tables stored in caldigit's RAID card!!!!
That means if the card died, notthing will get recovered.
Am I correct!!!! Maybe that is why a lot of my clients are complaining about the failure of caldigit's products.

No.

In CalDigit's case, the card is more basic. The Partition Tables are on the drives ONLY. If they're destroyed/damaged beyond what the level can tolerate, you're screwed. The card can't help you. But assuming the drives are fine (array is totally in tact), and you have a card failure, you should be able to swap it, and it find the array when attached.

But the experience I dealt with over the CalDigit card was a disaster. Constant drop-outs resulting in rebuilds, outright lost arrays,... soured me against them. Simply put, it was released to users far too prematurely, as the testing was no where near complete. More recent users have indicated they're fine now. But I just will NOT trust them for some time. Maybe if they change their habits with the next few cards, but I also am aware of the fact they don't design or manufacture their gear either, and that doesn't sit well with me either. Highpoint's the same way in this regard, and the tech support is lousy as a result.

In the case of the Areca's, you have BOTH the usual drive stored copy, but also a duplicate in the firmware. It allows recovery in situations that are otherwise toast. It's not fool proof (depends on what caused the failure), but it does help. I've seen it recover in situations that normally it's start over and restore. So simply put, it's an additional layer of "butt cover", and really nice to have.

 

[TheStrudel]

If you can manage to implement ZFS based systems on OS X (or Windows for that matter), then be my guest. I'm sure many would love the ability to skip an expensive controller (for an inexpensive HBA when more than 4 drives are needed). Me included.

But as that's not an option, and won't be, since Apple's dumped their attempt to make OS X ZFS compliant, we're all SOL.

My intuition tells me, based on everything I've heard, that 's just gotten tired of all that and decided to hire their own engineers and write their own replacement for HFS+. I wouldn't be surprised at all if it replicates some of the ZFS functionality we all want so badly, because  disdains hardware solutions that can be done more cheaply or flexibly in software.

We should all (well, those of us not well and fully invested in hardware solutions) hope that's true.

 

[joaoferro37]

Yes, I had the same experience on their too soon phased out products S2VR HD, S2VR Duo, FireWire VR, their SATA cards.

I feel like they were dying to gain market share and did a lot of forum spamming. However, consultant like us were helping customer to install caldigit's product got burned badly.

After tons of upset customers, I did my research and realized they did not even design and manufacture their own products. They had Accusys designed the card for CalDigit and I also heard Accusys is going out of business.

Since then, I switched from CalDigit to Areca and ATTO. One reason I saw a lot of "people" promoting CalDigit because CalDigit provides very healthy profit margin to resellers but again their products = Epic Fail and good luck with it.

 

[nanofrog]

My intuition tells me, based on everything I've heard, that 's just gotten tired of all that and decided to hire their own engineers and write their own replacement for HFS+. I wouldn't be surprised at all if it replicates some of the ZFS functionality we all want so badly, because  disdains hardware solutions that can be done more cheaply or flexibly in software.

We should all (well, those of us not well and fully invested in hardware solutions) hope that's true.

ZFS is definitely software based, and tecnically, so is any other form. It's just that with parity based levels, you need a proper controller that contains an NVRAM solution to the write hole issue. Otherwise, you'd get burnt at some point.

But I'm under the impression they'd like a newer file system, as would Windows. It's just really hard to do.

Yes, I had the same experience on their too soon phased out products S2VR HD, S2VR Duo, FireWire VR, their SATA cards.

I feel like they were dying to gain market share and did a lot of forum spamming. However, consultant like us were helping customer to install caldigit's product got burned badly.

After tons of upset customers, I did my research and realized they did not even design and manufacture their own products. They had Accusys designed the card for CalDigit and I also heard Accusys is going out of business.

Since then, I switched from CalDigit to Areca and ATTO. One reason I saw a lot of "people" promoting CalDigit because CalDigit provides very healthy profit margin to resellers but again their products = Epic Fail and good luck with it.

I hadn't heard Accusys was going under. That's really going to hurt CalDigit, but as I indicated, I'd never use their products without a major change in how they do things. And somehow, I don't think that's likely, given they don't do the work themselves (you need that hands-on aspect of testing at least, to gain the necessary experience proper customer support).

 

[joaoferro37]

ZFS is definitely software based, and tecnically, so is any other form. It's just that with parity based levels, you need a proper controller that contains an NVRAM solution to the write hole issue. Otherwise, you'd get burnt at some point.

But I'm under the impression they'd like a newer file system, as would Windows. It's just really hard to do.


I hadn't heard Accusys was going under. That's really going to hurt CalDigit, but as I indicated, I'd never use their products without a major change in how they do things. And somehow, I don't think that's likely, given they don't do the work themselves (you need that hands-on aspect of testing at least, to gain the necessary experience proper customer support).

It seems like CalDigit is using their customers as guinea pigs, having them test their products & then using the complaints/feedback/failures to try to improve their products.

 

[nanofrog]

It seems like CalDigit is using their customers as guinea pigs, having them test their products & then using the complaints/feedback/failures to try to improve their products.

That's the actual situation I was dealing with. The person that had the gear was the very first person to get it, and was a guinea pig. Days and days of testing, troubleshooting,... both on the phone and by email. Granted, they comp'ed some of the equipment, but in the end, it was totally worthless. It was dumped, and a completely new system was used. Card, enclosures, drives, and cables.