Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

OSx corruption of Raid-5 and Raid-6 arrays?

P

pprior

macrumors 65816
Original poster
Aug 1, 2007
1,448
9
I was inquiring as to buying a raid-6 array from a well known company. I received the following as part of a pre-sales discussion. I have never heard this before, but wanted to seek others opinions as it sounds just too bizarre to be true - I mean how can people run snow leopard server if this is the case and manage data?

He is recommending Raid-0 arrays backed up to another raid-0 array nightly, fwiw.

For the record, I'm looking at Atto hardware cards and mini-SAS 8 bay enclosures, not software raid.

Text of interest:

"These days a drive failure isn't the most common means of losing data. What we see the most problems with is what's called data corruption. It's when the data stored on a drive becomes scrambled for one of several reasons. On a software based RAID 0 it's usually easily repaired with tools like Disk Warrior or Tech Tools Pro. When the problem occurs on a Hardware based RAID 5 or RAID 6 the parity can quickly become unreadable making the RAID unusable by the computer. There are no tools that can repair the corruption on a parity RAID once it reaches a certain point. The problem arises because the parity data is stored across all of the drives, and in the case of a RAID 6 it has 2 sets of parity stored on each drive making corruption a huge problem.

If everything is going correctly and your workflow is correct you shouldn't see much corruption. We do see a lot occurring when video is stored on drives, and at times software or OS updates cause corruption all by themselves. OSX is famous for the problem since 10.5 came out. One common way to create problems for yourself is dragging and dropping files from one location to another. When you drag and drop you do not get the whole file, some of the index is lost. Do this enough and you'll have a problem because it's cumulative.

Try this; do a "Get Info" on a file and note the file size. Then drag that file to another location and do a "Get Info" on the destination file copy. You'll see that the new copy is smaller, it's lost some index data. Do this enough and the file can become unreadable and will have to be repaired. Do it enough on a parity RAID and you can make the file unreadable and irreparable."
 
pprior said:
He is recommending Raid-0 arrays backed up to another raid-0 array nightly, fwiw.
Click to expand...
Horrible advice.

I've never heard of what he's talking about with hardware implementations (software implementations are another story, but RAID 6 isn't offered in software - I don't recall ever seeing it anyway).
 
Yes, RAID 0 is for speed so why would a once-nightly backup array be RAID 0?

All that does is get you speed that you don't need but at the cost of doubling the likelihood of an unrecoverable drive failure in the backup array.
 
Well it also expands the drive size beyond what is physically available - I need more than one hardware drive will store, so it's not just for speed.

I'm well aware of the plusses and minuses of raid-0 arrays, but again this is a vendor mentioned widely in professional video editing forums and I was just really shocked to hear that statement of corruption occurring within Mac OS.

Nanofrog, do you think he might have been confusing software implementations of raid5? I don't know why because I was specifically asking about several Atto raid cards at the time.
 
pprior said:
Well it also expands the drive size beyond what is physically available - I need more than one hardware drive will store, so it's not just for speed.
Click to expand...
There's multiple ways to go though (JBOD, if you only need a large logical volume, and performance + capacity can be better handled by a redundant level of RAID).

pprior said:
...but again this is a vendor mentioned widely in professional video editing forums and I was just really shocked to hear that statement of corruption occurring within Mac OS.
Click to expand...
I've never heard of this. Assuming the premise is correct (problem with OS X), then it would occur with anything (single disk, or array, no matter the level or implementation).

So it sounds really wrong to me.

A link to the source could clarify some confusion.

BTW, is this from DigiLloyd?
I ask, as he may know his way around creative applications, but he doesn't have a proper understanding of RAID from what I've read on his site.

Placing the backup on the same disks as the primary data with the RAID 0 configuration he was running (just partitioned the disks, then created 2x stripe sets), which is beyond stupid.

Nor does he seem to understand the differences between RAID 5 and RAID 6 properly from what's been posted on the iStorage SAS Expander enclosure attached to an ATTO R348 post (lots of confusion, as well as condradictory statements in it).

I think he's learning, but slowly. And in the mean-time, the posts could cause others to risk their data by reproducing those configurations as they're assuming he knows what the heck he's doing.

Seem to recall he went back to consumer drives on that card, which were Hitachi DeskStar's of all things... another unwise decision (started with WD RE4's).

pprior said:
Nanofrog, do you think he might have been confusing software implementations of raid5? I don't know why because I was specifically asking about several Atto raid cards at the time.
Click to expand...
Possible, but I don't know the source of the information for sure.

If it's where I think it's from, it may be worse than not distinguishing the difference between a software and hardware implementation of RAID 5 (read the article I mentioned). He doesn't seem to have a true understanding of parity based arrays (doesn't realize that 5 and 6 are both parity, and have the same issues; n = 1 and n = 2 for RAID 5/6 respectively for fault tolerance is the only difference in implementation). Speed and capacity are affected as well, but I'm talking in terms of how the data is written (write hole exists for both levels, as well as nested parity 50/60).
 
No, it was not from digiloyd. It was from a vendor that regularly is advertised in creativecow forums and is well known for their products. It was in a private email after a sales inquiry, so I'm reluctant to post the vendor publicly, and was really more interested in the application/implications.

I don't truly have an understanding of how parity information is stored across multiple disks, especially in dual parity setups like raid-6, so I didn't feel confident calling BS on the man directly, but it just didn't pass the "sniff" test - i.e. if mac os truly had such a horrific bug in terms of data stability, I would have expected the PC boys to loudly trumpet it from the ramparts. On the other hand, he clearly stated it as fact and I'm not sophisticated enough to know for sure, hence my post here.
 
Bulls**t - OS X (10.5 or otherwise) does not "corrupt files" just by copying them. This is the most ridiculous piece of FUD I have seen recently. Go elsewhere with your business. The person that gave you that information is an idiot.
 
I'm with khollister on this one.... find a different consultant or vendor (not sure which this person is).

But using a consultant could offer the advantage of advice aimed at the best-fit solution or your needs rather than based on brand obligations (those that sell gear could have other factors influencing their recommendations than those that have no personal interest in the gear you'll end up purchasing).

Just a thought anyway. ;)
 
Last edited:
We're a vendor, and I've never offered any advice like that before--ever. That's the first I've heard of anyone recommending RAID-0 with a nightly backup to another RAID-0.
 
Hi pprior,

I would like to chime in too on this.

I have been in creative cow for some time too. One would think that macrumors would have less professional info but you would be surprised how knowledgable people are here, at least regarding hardware :eek:) nanofrog is very knowledgable regarding RAIDs!

Just to mention where I am coming from. I am writing this to you from a mac pro that has a RAID 5 (in hardware) from CalDigit (PCI proprietary internal bridge) for HD video, plus 5 Hard drives for apps, files and back ups. The system is running 10.6.5 and it also has a bunch of other hardware (a Kona 3 for video capture and a hacked PC ATI HD 5870 card). I have yet to experience data corruption at the level the salesman described.

Anyway, what I think happened is that the vendor may have had pushed for a sale without being technically fully informed, and he may have mixed his explanation somewhat.

If you do exactly what he says (copy the files between volumes and get info before and after the fact) the size information field MAY be different at the MB on disk level but it HAS to be exactly the same at the byte count level (the byte figure in parenthesis), otherwise you have a corrupted file copy, that is exactly what corruption means, the files are not the same. The difference arises from the fact that when volumes are of different sizes, the way the OS allocates each byte changes, the larger the volume capacity, the more the OS allocates bytes to "chunks" if you will, to each file. Try this simple experiment: Create a text file in text edit and save it with a single letter in it, or a single word. Get the info for it and you will see that the bytes on disk are different from the actual bytes in the file itself (from the get info box, it will say in the Size field something like 4KB on disk (315 bytes)). This means the OS allocated 4,000 or so bytes to a 315 byte file, because the size of the disk is so large it divides the "allocation chunks" to a fixed amount, and so 3,600 bytes are actually wasted.

Now, this phenomenon will increase as your volume size increases. However, this is independent of RAIDs. If you have 1 RAID 5 or 2 RAID 0s it won't make a difference regarding corruption at the allocation of space level. HOWEVER, everybody in this thread knows that running RAID 0s are very sensitive to a power failure or to be more precise, a write to volume in progress interruption of any kind. You do gain speed on those RAID 0s, but if your machine goes down for any reason while a write operation is in progress (power failure, kernel panic, network disconnect, etc.) and the parity write operation doesn't finish, you can kiss your volume goodbye. By the same token, if the operation does finish, you won't see any corruption going on. I don't know why he mentioned the index, it is not really critical to the discussion at hand.

To recap, if you repeat his steps as outlined in the quote, you will NOT see data corruption in ANY modern file system, especially involving a RAID one.

I won't point fingers to a particular vendor, but you can rest assured this isn't the case whoever the vendor is. BTW, my RAID is a CalDigit one and they do have a bad reputation around here. Even that one doesn't exhibit data corruption at any level.

The rule of thumb: go for RAID 0 for speed, RAID 1 for peace of mind, RAID 5 for a tradeoff of those too or RAID 6 for added peace of mind. If this is video you are working with, RAID 5 sounds the way to go. RAID 0 is just too risky.

Finally, you should ask Bob Zelin at the Creative Cow forum this same very question and see what he answers, I'd bet an apple t-shirt he would lecture you on RAIDs, the harsh way! :eek:P

Good luck and thanks for asking the question, it is the only way to learn! We all started that way!
 
mac666er said:
Anyway, what I think happened is that the vendor may have had pushed for a sale without being technically fully informed, and he may have mixed his explanation somewhat.
Click to expand...
Possible.

But the recommendation a stripe set as a safer alternative to a redundant level (parity based in this case) is ludicrous. So I'm more inclined to believe he didn't actually know what he was talking about (not trained properly, so I see it as worse than a bit confused, as following the advice would put a user's data in greater peril vs. any redundant level).
 
lots of great advice so far

one thing I would do is call back the company ask to speak to a sales manager ? ask them their thoughts see what they say then tell them what you were told by one of their people ? could be a new guy who has no clue ? again without knowing if its a big company or small ;) but might be good piece of mind
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back