Got a tip for us?
Let us know
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Which RAID format do you favor?
- Thread starter IceMacMac
- Start date
- Sort by reaction score
If you have good backups, no. However if you don't bother to back up, a 4 disk RAID 0 is an immensely terrible idea.
Not necessarily, as long as your maintaining a good backup strategy outside of the RAID.
The type of RAID I use really depends on what I'm trying to accomplish. Often, I find myself using RAID-10 (combines the benefits of RAID-0 and RAID-1, but requires at least 4 disks) over RAID-5.
There is a lot of debate with RAID-5/6 versus RAID-10. RAID-10 in most cases is faster because it doesn't have to deal with parity like RAID-5/6 arrays have to, but requires more physical disks to operate. It's a trade-off, really. But perhaps even less of trade-off these days, considering drives are so cheap now.
But you have the right idea already. RAID is good for redundancy in the event of a drive failure - a first line of defense, if you will. But it is no substitute for a sound backup strategy. Say perhaps you foolishly delete something on a mirrored RAID array. That delete operation just happened to the whole array and the file is gone...oops! I back up my important stuff to separate external hard drives and store them away.
Generally speaking, so long as you have a good good backup system in place, No.
As per the level used, it depends on what you're doing (i.e. you wouldn't want to use an RAID 0 for an OS/Applications location in a server
).What you need to understand about parity based arrays (5/6 and nested parity 50/60/51/61) is that it needs to be done via hardware due to the write hole issue associated with pairity based arrays (only able to solve it via hardware).
Unfortunately, there are software based RAID products out there that advertise RAID 5 operation (nothing more than a SATA card and drivers with RAID support). So users with this sort of product and using it for RAID 5 will get burnt (matter of when, not if) because it cannot handle the write hole issue at all.
I have a hardware RAID5. RAID5 gives you the most available storage and still have redundancy. I have 4x2TB drives in a RAID5 that gives me 6TB of usable space.
I was very intrigued when I first had RAID5 explained to me. It sounds almost magical.
But I've also heard that in some applications there are big performance penalties.
How do you generally use your system?
It's not magic, but it is a useful balance of redundancy and capacity. You do need a proper RAID controller to do it however.
For small files in high write conditions (entire file is smaller than a block/stripe size), it's not the best way go due to the read-modify-write implementation performed with both the data and parity blocks. So it tends not to be the best implementation for things like Random Access writes or Relational Databases.
Some cards have additional hardware to assist with this issue however, and is why the old "don't use it for databases" isn't always true anymore. But it still holds true with inexpensive products, such as RAID on a Chip products like the OWC Qx2.
What exactly are you trying to do?
The answer to the above question (and likely more to follow), will help get you aimed in the right direction, and in less time too.
Some cards have additional hardware to assist with this issue however, and is why the old "don't use it for databases" isn't always true anymore. But it still holds true with inexpensive products, such as RAID on a Chip products like the OWC Qx2.
What exactly are you trying to do?
The answer to the above question (and likely more to follow), will help get you aimed in the right direction, and in less time too.
You hit upon a magic note. I am using the Qx2...so though it offers RAID 5 as an option, I'll avoid it. No point in complicating the matter and risking a performance hit, for protection that might not be so sure.
I do tons of AE and c4d animation...so I produce lots of big files. I think I'll go with RAID 0 and have two sets of backups.
Another variable to consider is the use of enterprise-grade drives. These are designed to operate in more demanding applications like RAID arrays running 24/7. They are less susceptible to heat and vibration-related problems and typically have lower unrecoverable read error rates.
I know you already have some drives but it is something to consider in a production environment esp if you are using RAID 0.
I know you already have some drives but it is something to consider in a production environment esp if you are using RAID 0.
I use my RAID5 Array as a general file store on my OSX server. Built for sheer capacity and redundancy. Speed was not necessarily an issue for me, although I am storing my Movie collection which streams to a couple of Macs around my home. It is plenty fast enough for that. While RAID5 would obviously not as fast as a RAID0 solution, it is faster than a RAID1.
The Qx2 makes a great backup location. For primary storage, it can under rather limited conditions IMO (where redundancy is more important than speed).
But at least it's RAID 5 is hardware, so your data won't fall victim to the write hole issue as it would under a software RAID 5 implementation.
In your case, you won't hit the specific performance limitations of RAID 5 listed previously, as you're creating large files (not entire files that are smaller than the stripe size = when you would run into those issues).
Now the real question; are you a professional (earning a living making your AE and C4D files)? or are you a hobbyist/enthusiast/student?
The reason for this, is RAID 0 is cheap, but with the trade-off of a massive amount of time when a failure occurs (you'd have to put in the time to find the bad disk, replace it, re-initialize the array, then restore the data off of the backups). Once that's done, you'll have to check to see what work was lost (done after the last backup, but before the array failed). All that time adds up.
Now if you're in the hobbyist/enthusiast/student category, this is likely fine, as it generally means the time is available, and the budget is rather tight.
But if you're a professional earning a living with this machine, it's not the way to go at all. You'd really need to take your time (and avoidance of lots of aggravation) into account, and implement a proper hardware RAID system of some sort (separate disks too, such as dedicated boot and scratch locations; not multiple partitions).
For example:
- SSD for OS/applications (attach it to the system's ICH for boot)
- SSD dedicated to scratch
- n * HDD's in an array (parity gives a nice balance of capacity, speed, and redundancy)
Now if you do go with a proper RAID solution, you will need to use Enterprise Grade disks (not an option, as it has to do with the recovery timings are done differently, and it's part of the disk's firmware; consumer models won't be stable, if you can even get them to initialize).
You'd also need to get an enclosure/internal mounting system, or possibly both. As well as run all of this on a proper UPS (pure sine wave inverter), and if possible, a Battery Backup Unit for the RAID card.
You could use the Qx2 as your backup location, so you've got that part already if you need to consider such an implementation.
Now all of this isn't cheap, but it will improve your workflow, and allow you to get more work in a period of time (i.e. instead of 4x jobs per month, you may be able to get 6). Thus increasing your profits by more than what the equipment costs. So there's a financial incentive to spend funds on such a setup as well (increases your anual profits by making your work more efficient).
Just something to consider at least.
The Qx2 makes a great backup location. For primary storage, it can under rather limited conditions IMO (where redundancy is more important than speed).
But at least it's RAID 5 is hardware, so your data won't fall victim to the write hole issue as it would under a software RAID 5 implementation.
In your case, you won't hit the specific performance limitations of RAID 5 listed previously, as you're creating large files (not entire files that are smaller than the stripe size = when you would run into those issues).
Now the real question; are you a professional (earning a living making your AE and C4D files)? or are you a hobbyist/enthusiast/student?
The reason for this, is RAID 0 is cheap, but with the trade-off of a massive amount of time when a failure occurs (you'd have to put in the time to find the bad disk, replace it, re-initialize the array, then restore the data off of the backups). Once that's done, you'll have to check to see what work was lost (done after the last backup, but before the array failed). All that time adds up.
Now if you're in the hobbyist/enthusiast/student category, this is likely fine, as it generally means the time is available, and the budget is rather tight.
But if you're a professional earning a living with this machine, it's not the way to go at all. You'd really need to take your time (and avoidance of lots of aggravation) into account, and implement a proper hardware RAID system of some sort (separate disks too, such as dedicated boot and scratch locations; not multiple partitions).
For example:
- SSD for OS/applications (attach it to the system's ICH for boot)
- SSD dedicated to scratch
- n * HDD's in an array (parity gives a nice balance of capacity, speed, and redundancy)
Now if you do go with a proper RAID solution, you will need to use Enterprise Grade disks (not an option, as it has to do with the recovery timings are done differently, and it's part of the disk's firmware; consumer models won't be stable, if you can even get them to initialize).
You'd also need to get an enclosure/internal mounting system, or possibly both. As well as run all of this on a proper UPS (pure sine wave inverter), and if possible, a Battery Backup Unit for the RAID card.
You could use the Qx2 as your backup location, so you've got that part already if you need to consider such an implementation.
Now all of this isn't cheap, but it will improve your workflow, and allow you to get more work in a period of time (i.e. instead of 4x jobs per month, you may be able to get 6). Thus increasing your profits by more than what the equipment costs. So there's a financial incentive to spend funds on such a setup as well (increases your anual profits by making your work more efficient).
Just something to consider at least.
I'm a 15 year veteran professional.
I don't overtly disagree with you on your plan as an ideal, but the nature of my business spreads my money in a lot of different directions:
--Software. Upgrading Adobe Master Suite every year, Vue 3d, C4d, VRay, Unity 3D, plugins, etc. etc.;
--Photography. My Canon 1D Mark IV and never ending lenses;
--Video (I plan to buy a Red camera)
--Multiple computer systems for my internal render farm.
With that in mind I figured 4 drives at RAID 0 in an affordable QX2 would get me speed and then I'd have use older drives for onsite and offsite backups.
Maybe next year I can go for NAS drives over Thunderbolt and I'll go for enterprise drives and a pricier enclosure. This year: It's all about the Red.
I will jump in on this.
I am going to say RAID 5 hands down. RAID-0 has its place, but since the risk of data loss just compounds with its use and size (i.e. the more drives you use, the riskier it gets, and the longer you use it, the riskier it gets!) for a benefit in speed, I am going to say, you can't put in valuable data in there.
By valuable data, I mean data not easily replaceable. Sure, you can put in videos, applications, data in general if you are absolutely certain you can just install or copy everything again on it, and DON'T mind going through all the effort of replacing it.
Now, and this is where it gets interesting, we all are responsible for our own data requirements and security. And on top of all technical discussions that have popped up in this thread, I would argue that data backup goes beyond technical and it requires some thought, even consider human error (as it happened to me actually).
In this very forum, I have seen people asking for help trying to recover important data from a failed RAID-0 by trying to revive RAID drives that are acting up. And the reason I mention this, is because data catastrophes are not that rare.
I will include myself in as a victim to this. I have a RAID-5 volume that backs up with time machine. I also copy this volume to two other different media, one of which gets backed up too. In essence, 4 backups of the same stuff, without including the RAID.
Crazy, I know, but talk to a data security expert and if he is worth his salt he will argue you can never be too paranoid about this. And during this year I have come to experience the wrath of data gone awry.
I use my RAID for video. I shot probably a hundred shots back in 2007 and 2008 that I knew I had in storage. Hours of shots that would take hours to re-shoot, but are also unique in their own right. One project comes along that needs them, so I say no problem, search my RAID, they are not there. I search the two volumes that I copy them manually to, they are not there. I search time machine, they are not there.
After some further investigation and now starting really to realize they must be really gone, I realize my footage was overwritten by myself with another folder with proxy files (files with the same name, that were pretty much worthless). I check all my volumes again and the same issue happened to all of them. I check time machine and it had run out of space back when it happened, so it didn't keep the back up from way back in 2008.
So, just holding to my last string of hope I checked my last back up drive, and sure enough, it had backed up all this data and was all set
Long story short, never be too paranoid about back ups and redundancy. Hard drives are VERY cheap now.
The key question is, can you live without the data in your RAID? And I would argue, even if you are not putting in your iPhoto library full of pictures from years, it is still a disaster waiting to happen. What if you just use it for Photoshop and you have 3 hrs worth of work there and then it goes belly up? Or any other program? Is it worth it to do a re-install of X hrs of whatever you had there? Even if you don't need to reinstall, is it worth it to have X hrs of work lost there? You mentioned RED, what if your RAID fails and you lose your working volume with whatever footage plus work-in-progress AND you can't use the volume?
If you need speed so that RAID-0 becomes attractive, and I know RED full resolution certainly does need top speed, I would argue, why don't make the sacrifice and spend a little more on RAID-5? It is added insurance and peace of mind.
I am going to say RAID 5 hands down. RAID-0 has its place, but since the risk of data loss just compounds with its use and size (i.e. the more drives you use, the riskier it gets, and the longer you use it, the riskier it gets!) for a benefit in speed, I am going to say, you can't put in valuable data in there.
By valuable data, I mean data not easily replaceable. Sure, you can put in videos, applications, data in general if you are absolutely certain you can just install or copy everything again on it, and DON'T mind going through all the effort of replacing it.
Now, and this is where it gets interesting, we all are responsible for our own data requirements and security. And on top of all technical discussions that have popped up in this thread, I would argue that data backup goes beyond technical and it requires some thought, even consider human error (as it happened to me actually).
In this very forum, I have seen people asking for help trying to recover important data from a failed RAID-0 by trying to revive RAID drives that are acting up. And the reason I mention this, is because data catastrophes are not that rare.
I will include myself in as a victim to this. I have a RAID-5 volume that backs up with time machine. I also copy this volume to two other different media, one of which gets backed up too. In essence, 4 backups of the same stuff, without including the RAID.
Crazy, I know, but talk to a data security expert and if he is worth his salt he will argue you can never be too paranoid about this. And during this year I have come to experience the wrath of data gone awry.
I use my RAID for video. I shot probably a hundred shots back in 2007 and 2008 that I knew I had in storage. Hours of shots that would take hours to re-shoot, but are also unique in their own right. One project comes along that needs them, so I say no problem, search my RAID, they are not there. I search the two volumes that I copy them manually to, they are not there. I search time machine, they are not there.
After some further investigation and now starting really to realize they must be really gone, I realize my footage was overwritten by myself with another folder with proxy files (files with the same name, that were pretty much worthless). I check all my volumes again and the same issue happened to all of them. I check time machine and it had run out of space back when it happened, so it didn't keep the back up from way back in 2008.
So, just holding to my last string of hope I checked my last back up drive, and sure enough, it had backed up all this data and was all set
Long story short, never be too paranoid about back ups and redundancy. Hard drives are VERY cheap now.
The key question is, can you live without the data in your RAID? And I would argue, even if you are not putting in your iPhoto library full of pictures from years, it is still a disaster waiting to happen. What if you just use it for Photoshop and you have 3 hrs worth of work there and then it goes belly up? Or any other program? Is it worth it to do a re-install of X hrs of whatever you had there? Even if you don't need to reinstall, is it worth it to have X hrs of work lost there? You mentioned RED, what if your RAID fails and you lose your working volume with whatever footage plus work-in-progress AND you can't use the volume?
If you need speed so that RAID-0 becomes attractive, and I know RED full resolution certainly does need top speed, I would argue, why don't make the sacrifice and spend a little more on RAID-5? It is added insurance and peace of mind.
I will jump in on this.
I am going to say RAID 5 hands down. RAID-0 has its place, but since the risk of data loss just compounds with its use and size (i.e. the more drives you use, the riskier it gets, and the longer you use it, the riskier it gets!) for a benefit in speed, I am going to say, you can't put in valuable data in there.
By valuable data, I mean data not easily replaceable. Sure, you can put in videos, applications, data in general if you are absolutely certain you can just install or copy everything again on it, and DON'T mind going through all the effort of replacing it.
Now, and this is where it gets interesting, we all are responsible for our own data requirements and security. And on top of all technical discussions that have popped up in this thread, I would argue that data backup goes beyond technical and it requires some thought, even consider human error (as it happened to me actually).
In this very forum, I have seen people asking for help trying to recover important data from a failed RAID-0 by trying to revive RAID drives that are acting up. And the reason I mention this, is because data catastrophes are not that rare.
I will include myself in as a victim to this. I have a RAID-5 volume that backs up with time machine. I also copy this volume to two other different media, one of which gets backed up too. In essence, 4 backups of the same stuff, without including the RAID.
Crazy, I know, but talk to a data security expert and if he is worth his salt he will argue you can never be too paranoid about this. And during this year I have come to experience the wrath of data gone awry.
I use my RAID for video. I shot probably a hundred shots back in 2007 and 2008 that I knew I had in storage. Hours of shots that would take hours to re-shoot, but are also unique in their own right. One project comes along that needs them, so I say no problem, search my RAID, they are not there. I search the two volumes that I copy them manually to, they are not there. I search time machine, they are not there.
After some further investigation and now starting really to realize they must be really gone, I realize my footage was overwritten by myself with another folder with proxy files (files with the same name, that were pretty much worthless). I check all my volumes again and the same issue happened to all of them. I check time machine and it had run out of space back when it happened, so it didn't keep the back up from way back in 2008.
So, just holding to my last string of hope I checked my last back up drive, and sure enough, it had backed up all this data and was all set
Long story short, never be too paranoid about back ups and redundancy. Hard drives are VERY cheap now.
The key question is, can you live without the data in your RAID? And I would argue, even if you are not putting in your iPhoto library full of pictures from years, it is still a disaster waiting to happen. What if you just use it for Photoshop and you have 3 hrs worth of work there and then it goes belly up? Or any other program? Is it worth it to do a re-install of X hrs of whatever you had there? Even if you don't need to reinstall, is it worth it to have X hrs of work lost there? You mentioned RED, what if your RAID fails and you lose your working volume with whatever footage plus work-in-progress AND you can't use the volume?
If you need speed so that RAID-0 becomes attractive, and I know RED full resolution certainly does need top speed, I would argue, why don't make the sacrifice and spend a little more on RAID-5? It is added insurance and peace of mind.
Great post!
Now the rub: What is the performance comparison between Raid 0 and 5? Are we talking 2-3 times variance?
I realize that funds have to be used for other areas, such as software and other equipment, but I wouldn't recommend just looking at a redundant RAID system as an ideal/something that can be put off until some later date, given what you've listed here.
As mac666er pointed out, the danger of loosing irrecoverable data or the time spent to restore it from backup media can't be discounted. Not just in aggravation, but financially speaking due to the time spent that could have been used for billable hours (very real impact on your bottom line).
Now if you consider that you'd also be freed up to do more work per month (or whatever unit you prefer), you can further increase your bottom line per unit time. This is where the financials work out in your favor, and additional profit is something that's very understandable and desired (real money, not some sort of Fun Bucks that can only be spent at one store and only during a blue moon sale
).In your case, a RAID 5 would be the bare minimum (acceptable as you'll be in front of the system in order to catch a failure immediately for the most part).
So if at all possible, it's really in your best interests to go ahead and find a way to do more than a stripe set for your data NOW. Not later (especially with importing RAW footage from the camera, as I'd doubt you want to have to repeat that due to a dead stripe set).
For what you're doing, the Qx2 isn't the best way to go, particularly in a stripe set.
As per a Thunderbolt solution, you'd get better throughputs and very likely lower costs by using a PCIe based RAID card (8x lanes are definitely capable of moving more data than TB is). Using PCIe gen 2, 8x lanes can move 4GB/s of data. TB is only good for 1.25GB/s of data, and that's assuming no latency or overhead (need to see real world results).
Given what's been listed, I'd say so as well.
Absolutely. The importance of data security (backups and everything else) cannot be stressed enough.
It's amazing to see how often I see independent pros and SMB's that have no backup system in place, or what is, is useless (i.e. manual settings, and it's never done constently, due to something like having to dig out an external disk and plug it in, or feed optical disks).
Not much at all really (~15% slower for RAID 5).
- Stripe set performance = n disks * performance of a single disk
- RAID 5 ~= n disks * performance of a single disk * (0.85)
- Stripe set = 10 * 100MB/s = 1GB/s
- RAID 5 = 10 * 100MB/s * 0.85 = 850MB/s
And with the right card with enough write cache, you can narrow the margin even further.
Only 15%? That sounds pretty good. I purchased the QX2 and some very affordable 3TB drives two weeks ago so that's water under the bridge. But I can migrate that 12TB monster to serve as a backup solution in six to twelve months.
15% hit is much less than I expected and would be very workable. I might reformat the RAID as 5.
Last edited:
I don't want to add undue paranoia to the discussion, but I think everyone will agree that Raid 5, 6, 10, 50 whatever... is not a replacement for backup.
From personal experience, I can tell you that Raid 5, while better that Raid 0 as far as redundancy goes, is NOT fail proof. I had an array on a server some years ago that had a drive fail. No big deal, SCA 80 pin SCSI, hot swap the cold shelf spare in... rebuild begins. During the rebuild a 2nd drive failed and the entire array was lost. Chances of that happening are quite slim, but it can happen and having a Raid-5 (or Raid-6 even) array is still not a substitute for good data backup. My company learned that the hard way, circa 2002.
Just my 2 cents worth...
From personal experience, I can tell you that Raid 5, while better that Raid 0 as far as redundancy goes, is NOT fail proof. I had an array on a server some years ago that had a drive fail. No big deal, SCA 80 pin SCSI, hot swap the cold shelf spare in... rebuild begins. During the rebuild a 2nd drive failed and the entire array was lost. Chances of that happening are quite slim, but it can happen and having a Raid-5 (or Raid-6 even) array is still not a substitute for good data backup. My company learned that the hard way, circa 2002.
Just my 2 cents worth...
RAID 5 makes a really good level for video editing and animation (large files), especially for a professional earning a living with their systems/equipment. It's a far better way to go than a stripe set, and cheaper than 10 for any large capacity over 6TB (based on 4x 3TB disks; 5 is faster than a 10 configuration, though not quite as redundant = trade-off between these two levels).
Just look around MR, and see what others that have done for their video editing storage systems (those that went past a stripe set, and more importantly, those that got burnt with stripe sets, then went to RAID 5 or better).
I see that you quoted part of my post before I edited/changed my thinking. Once I saw that the performance penalty was nominal...you all have convinced me.
As for viewing RAID 5 as a "backup"...no I wouldn't do that. There will still be onsite and offsite backups.
I still can't get over how many great posters there on this forum!