Advice on internal disks

[nanofrog]

Wow, thanks Nano! That was nice (and thorough) explanation. It seems to put all the technicals in place. Of course that leaves me with the general question: When would you recommend using RAID class drives?

Generally speaking, anytime a RAID array is used as a primary data storage area, and especially if the system is intended for high availability (time online, no faults/failures, such as 24/7). There's exceptions of course, such as an external backup that isn't always in use.

Regardless of the wimpier self-recovery are they going to act pretty much the same for the same length of time (or longer) than non RAID class drives in a single drive system and/or in a RAID array. I mean other than being faster of course.

It's not that the single drive system is wimpy, it's designed for a single drive. So it isn't as extensive as what's needed for a multiple drive array.

Consumer drives typically have a UBE (Unrecoverable Bit Error) of 1E14, while an enterprise drive has a rating of 1E15. That order of magnitude can make a significant difference, and may be usefull as a single drive, if the data is critical. Far cheaper than a data recovery system, and the time (in terms of $$ value) needed to reconstruct an array from backups.

Consumer drives aren't really meant for RAID. They can't take it, as they weren't designed for that use. (Lack of vibration tollerance, lower UBE and MTBF on the reliability side, and a few features aimed at improved throughput such as RAFF). They can be be used, but at a higher risk of failure and in some cases, instability. That's the compromise for lower cost. And in some cases, hardware RAID in particular, consumer drives may not work. Software RAID has better luck as far as compatibility issues, but you still have the lower reliability issues.

In WD's case, some of thier consumer models do offer an increased UBE (1E15), but they still don't have the vibrational tolerance and the like. These are candidates for TLER modification to get lower the cost. Not quite as much of a compromise with the improved UBE, and can be usefull, such as a backup or other system that doesn't require 24/7 operation, but the data is highly important.

It can be a hard call, and is always dependant on the specifics IMO.

 

[Tesselator]

Hmm, OK. I'm doing my best to read between the lines here and I'm getting some signals that for a typical MP user (Joe Apple-Pack) It doesn't matter almost at all: The RAID class drives work just as good for single drives and of course better in a RAID array. Consumer drives also work both ways as well - especially if the RAID is a software RAID - at least I've never heard of two or more identical drives not working in a SW-RAID. Actually I've never heard of consumer drives not working in a hardware RAID - but that just means I haven't heard of it. I see manufacturers warnings and recommendations accompanying some high-end RAID controllers.

Is that a fair generalization? Or did I misunderstand the importance of some of the points you made?

Thanks for yet another excellent reply too! I think you're my new hero! Hehehe: Up, Up, and AWAY!!! NanoFrog... But seriously, thanks man! I dig learning!

 

[Mac Husky]

I join Tesselator´s thanks to your detailed explanations, nanofrog.

Having a look at the costs in Germany:
Samsung Spinpoint F1 1TB: € 70,-
Samsung F1 RAID class 1TB: € 110,-
Western Digital Caviar Black 1TB: € 95,-
Western Digital RE3 1TB: € 135,-

If the 1TB isn´t really neaded and the price hurts a lot,
I would go for 750 or 500 GB drives to lower the costs that way
but go for the RAID class versions instead when using RAID0/1.

Still have a question unanswered in another thread:
Planning a VelicoRaptor 300 as system disk - is it usefull just to let it be a system disk and not to integrate it into the RAID system by making a second partition for that?!

I would go with the VR for system and apps and another 3 more disks probably using RAID class versions.
Building RAID0 for partitions of disks 2+3 (scratch disk for Photoshop) and RAID1 for the other stuff on 2+3.
Having a big drive in slot 4 using as internal backup for the other 3 disks (Time Machine).
Doing another external backup for disk 4.

 

[Lordedmond]

Having work in the IT sector for many years when i worked , It is a cautionary point regarding disk array's


two drive in an array will halve the MTBF ie the bath tub curve gets narrower
four drives and the MTBF falls to a quarter of a single drive


its a pure mathematical thing more drive's more things to fail

 

[Tesselator]

Not to seem too challenging and for sure not meant personally but I call BS. If it is even true that math applies to this it's like the math that shows a bumblebee can't fly.

The actual stats just do not show this to be true. I set-up and managed a VOD system for a friend of mine who owns a hotel chain. There were 4 servers with 16 drives in each RAID for the first 3 years at each of the 4 hotels. Zero failures for 3 years. Yet that math would seem to indicate I should have seen a failure every two months or something. For year 4~6 (almost another 3 years) there were 8 drives per server, 4 servers per hotel, 4 hotels. and there was one failure so far but the server itself smoked so maybe the failure was cause by the PSU that died. All those drives were consumer grade I think, and running 24/7 - it was VOD after all. Totally that's 256 drives for 3 years and 128 of them for the following 3.

At the two universities I taught at for a period of 8 years I managed the systems in 2 classrooms at one uni and 1 classroom at the other. All three had 40 machines each. That's another 120 drives in the student's machines alone. We got new machines every 3 years so I dunno what would have happened but by that ever so popular math that I hear so often at least one of them should have failed. None did. There was a problem with a few motherboards and I remember an LCD that had to be replaced.

I know that it's usually one of two things that kills a drive. Heat or vibration. So I made sure all these systems ran cool and had a solid steady base. No wooden floors, no desktops, and extra fans if needed.

But I do keep hearing about this probability factor thingy...


PS: That's not even counting all the drives I'v owned for my SOHO systems - which for my career has been rather a lot over the past 20 years.


HDD's simply do NOT change their MTBF rating just because you have more then one. If treated right modern drives last for as long as their rating - and that's all there is to it. Now if QC is lax you might see one fail in the 1st week or something but other than that given the conditions are good no such accumulative probabilities apply.


.

 

[nanofrog]

Hmm, OK. I'm doing my best to read between the lines here and I'm getting some signals that for a typical MP user (Joe Apple-Pack) It doesn't matter almost at all: The RAID class drives work just as good for single drives and of course better in a RAID array. Consumer drives also work both ways as well - especially if the RAID is a software RAID - at least I've never heard of two or more identical drives not working in a SW-RAID. Actually I've never heard of consumer drives not working in a hardware RAID - but that just means I haven't heard of it. I see manufacturers warnings and recommendations accompanying some high-end RAID controllers.

Is that a fair generalization? Or did I misunderstand the importance of some of the points you made?

Thanks for yet another excellent reply too! I think you're my new hero! Hehehe: Up, Up, and AWAY!!! NanoFrog... But seriously, thanks man! I dig learning!

As a generalization, Yes. For the issues with compatibility and hardware controllers, take a look at some of the high end SAS offerings from Areca and Atto. SAS is picky, and tends to only work with enterprise drives, and not even all of them. Check Areca's HDD Compatibility List for the ARC-1680 series and ARC-12xx (SAS) models, and you'll see what I'm talking about.

The ARC-1680s won't work with WD RE3's, but will with the RE2's. Their SATA controllers offer a better scenario, and can run at least some consumer models as well.

The TLER utility allows you to make at least the GP series drives work with the SAS controllers, but they won't work without the adjusted values.

I join Tesselator´s thanks to your detailed explanations, nanofrog.

Having a look at the costs in Germany:
Samsung Spinpoint F1 1TB: € 70,-
Samsung F1 RAID class 1TB: € 110,-
Western Digital Caviar Black 1TB: € 95,-
Western Digital RE3 1TB: € 135,-

If the 1TB isn´t really neaded and the price hurts a lot,
I would go for 750 or 500 GB drives to lower the costs that way
but go for the RAID class versions instead when using RAID0/1.

I like the smaller drives, and went with 320GB RE3's for my SATA array (Backup). But I'm paranoid about my data, and I've seen a few too many bills/estimates from Data Recovery companies. So to me, the cost is justified, as is a really good backup system. It's cheaper by a substantial amount if you find yourself in need of such services.

Still have a question unanswered in another thread:
Planning a VelicoRaptor 300 as system disk - is it useful just to let it be a system disk and not to integrate it into the RAID system by making a second partition for that?!

I would go with the VR for system and apps and another 3 more disks probably using RAID class versions.
Building RAID0 for partitions of disks 2+3 (scratch disk for Photoshop) and RAID1 for the other stuff on 2+3.
Having a big drive in slot 4 using as internal backup for the other 3 disks (Time Machine).
Doing another external backup for disk 4.

You can use a separate (single) disk for OS if you wish. It has the advantage of booting faster, and is necessary in some cases. Such as using a hardware controller that won't boot OS X, or perhaps any OS at all.

Booting off the RAID controller (hardware) gives you the benefit of improved throughput for everything on the array, and perhaps redundancy, depending on the type of array used. But the card's initialization (loading firmware) time can actually slow down the time needed to boot. Once up however, you'll get better throughput.

Having work in the IT sector for many years when i worked , It is a cautionary point regarding disk array's

two drive in an array will halve the MTBF ie the bath tub curve gets narrower
four drives and the MTBF falls to a quarter of a single drive

its a pure mathematical thing more drive's more things to fail

I think I know what you're referring to, but it's not MTBF that changes, but rather the UBE due to additional drives/platters. UBE (array) = UBE(single drive)/(n drives), and is most applicable to RAID 0, as it has no redundancy. It's over simplified for other implementations. Take into account redundancy, usage patterns, including availability, and it gets complicated. I've not seen any mathematical model that can accurately predict the lifespan with absolute accuracy. Only estimates, as the data used are estimates.

The actual stats just do not show this to be true. I set-up and managed a VOD system for a friend of mine who owns a hotel chain. There were 4 servers with 16 drives in each RAID for the first 3 years at each of the 4 hotels. Zero failures for 3 years. Yet that math would seem to indicate I should have seen a failure every two months or something. For year 4~6 (almost another 3 years) there were 8 drives per server, 4 servers per hotel, 4 hotels. and there was one failure so far but the server itself smoked so maybe the failure was cause by the PSU that died. All those drives were consumer grade I think, and running 24/7 - it was VOD after all. Totally that's 256 drives for 3 years and 128 of them for the following 3.

At the two universities I taught at for a period of 8 years I managed the systems in 2 classrooms at one uni and 1 classroom at the other. All three had 40 machines each. That's another 120 drives in the student's machines alone. We got new machines every 3 years so I dunno what would have happened but by that ever so popular math that I hear so often at least one of them should have failed. None did. There was a problem with a few motherboards and I remember an LCD that had to be replaced.

Then you should know what I'm talking about.

I know that it's usually one of two things that kills a drive. Heat or vibration. So I made sure all these systems ran cool and had a solid steady base. No wooden floors, no desktops, and extra fans if needed.

But I do keep hearing about this probability factor thingy...


PS: That's not even counting all the drives I'v owned for my SOHO systems - which for my career has been rather a lot over the past 20 years.


HDD's simply do NOT change their MTBF rating just because you have more then one. If treated right modern drives last for as long as their rating - and that's all there is to it. Now if QC is lax you might see one fail in the 1st week or something but other than that given the conditions are good no such accumulative probabilities apply.
.

The drive's MTBF doesn't change. It has to do with the platter density and platter quantity combined with the UBE. As arrays grow in capacity, and number of drives, you may find yourself in a bad situation. For example, if you have a RAID 5, say 12 or more drives, with a capacity of 10TB+ on consumer drives. You get a drive failure, and replace a drive. It starts to rebuild. Only you now have another drive fail during the rebuild. You're array is DOA. Fix it, and restore from backups. Takes quite a bit of time, and will likely experience the same failure if you don't replace all the drives. Here's where the enterprise drives can save you, as the order of magnitude would prevent the failure. If allowed to go on long enough, they'd do the same thing, but it takes longer. To me, the enterprise models are sort of a cheap form of insurance.

So watching the array sizes and creating a replacement schedule is about the best way to combat such a disaster (primary array only, backups are further insurance against data loss). UBE's aren't keeping up with the capacity/density increases with drives.

 

[Mac Husky]

You can use a separate (single) disk for OS if you wish. It has the advantage of booting faster, and is necessary in some cases. Such as using a hardware controller that won't boot OS X, or perhaps any OS at all.

Booting off the RAID controller (hardware) gives you the benefit of improved throughput for everything on the array, and perhaps redundancy, depending on the type of array used. But the card's initialization (loading firmware) time can actually slow down the time needed to boot. Once up however, you'll get better throughput.

Thx. I decided not to go for a hardware RAID. In another thread, there was a question of the user DrawingArt, if boot and software on RAID0 would be much faster then working files on RAID0? And he asked, of which he would have more benefit.

That is leading me to the following:

I planned a WD VelociRaptor for system and apps disk so far and another 2 disks (WD RE3) going RAIDO with my data as I always thought, that leaving system and apps alone on one disk is usefull when possible - as you agreed with for speed reasons! Would abough thoughts lead me to buy another WD VelociRaptor for Position 2 and set it in RAID0 with the first one - including system, apps and files/data? Or even let the WD VelociRaptor go and buy RE3 as disk 1+2 perhaps 3 additionally to put them into RAIDO all together?! I so far avoided to stretch system and apps over more than one drive. Didn´t thought it would make sense to build a RAIDO for system and apps to increase the speed. Planning RAIDO for data only.

What would you say should one prefer if both settings might be practicable:

First way:
Disk 1: WD VelociRaptor installing system and apps only
Disk 2+3: WD RE3s with some date RAID0 and other RAID1 if needed
Disk 4: WD Caviar Black 1TB (or even bigger) for fully backup (system, apps, data) via Time Machine

Second way:
Disk 1+2: WD VelociRaptors or RE3s summing-up system, apps and data in a RAIDO and maybe some data in RAID1 if needed
Disk 3: WD Caviar Black 1TB (or even bigger) for fully backup (system, apps, data) via Time Machine

 

[Tesselator]

As a generalization, Yes.

Thanks man! You and others like you are what make forums so damn cool! Seriously! All outstanding information!

Then you should know what I'm talking about.

Yeah. I taught CG art and animation though. And I'm always willing to learn more than I think I know.


.

 

[nanofrog]

Thx. I decided not to go for a hardware RAID. In another thread, there was a question of the user DrawingArt, if boot and software on RAID0 would be much faster then working files on RAID0? And he asked, of which he would have more benefit.

That is leading me to the following:

I planned a WD VelociRaptor for system and apps disk so far and another 2 disks (WD RE3) going RAIDO with my data as I always thought, that leaving system and apps alone on one disk is usefull when possible - as you agreed with for speed reasons! Would abough thoughts lead me to buy another WD VelociRaptor for Position 2 and set it in RAID0 with the first one - including system, apps and files/data? Or even let the WD VelociRaptor go and buy RE3 as disk 1+2 perhaps 3 additionally to put them into RAIDO all together?! I so far avoided to stretch system and apps over more than one drive. Didn´t thought it would make sense to build a RAIDO for system and apps to increase the speed. Planning RAIDO for data only.

What would you say should one prefer if both settings might be practicable:

First way:
Disk 1: WD VelociRaptor installing system and apps only
Disk 2+3: WD RE3s with some date RAID0 and other RAID1 if needed
Disk 4: WD Caviar Black 1TB (or even bigger) for fully backup (system, apps, data) via Time Machine

Second way:
Disk 1+2: WD VelociRaptors or RE3s summing-up system, apps and data in a RAIDO and maybe some data in RAID1 if needed
Disk 3: WD Caviar Black 1TB (or even bigger) for fully backup (system, apps, data) via Time Machine

From what I understand of graphics/video use, you're "First Way" would be better IMO. Put the scratch data on the RAID0, as this is what I understand is the weak spot for speed.

Tesselator (or anyone else), please feel free to chime in, as I don't use these types of applications.

As for using disks 2+3 for both a RAID 0 and a RAID 1, I wouldn't do it. If anything went wrong with one of the drives, both arrays are affected. Just a rebuild for the RAID 1, but a complete reinstallation for RAID 0.

You can add another pair of drives in the empty optical bay, assuming you didn't order a second optical drive. Even if you did, just place it into an external enclosure, as external RAID enclosures are more expensive. For mounting, solutions exist, though not inexpensive. You could also DIY a mount from an old CD/DVD drive. Just dissasemble it, and use one of the metal plates from the enclosure. Just drill some holes.

Thanks man! You and others like you are what make forums so damn cool! Seriously! All outstanding information!

Yeah. I taught CG art and animation though. And I'm always willing to learn more than I think I know.

I stick more with hardware, as that's where my income is derived. Though I've interests that I'd like to learn more about as well.

 

[Tesselator]

From what I understand of graphics/video use, you're "First Way" would be better IMO. Put the scratch data on the RAID0, as this is what I understand is the weak spot for speed.

Tesselator (or anyone else), please feel free to chime in, as I don't use these types of applications.

As for using disks 2+3 for both a RAID 0 and a RAID 1, I wouldn't do it. If anything went wrong with one of the drives, both arrays are affected. Just a rebuild for the RAID 1, but a complete reinstallation for RAID 0.

You can add another pair of drives in the empty optical bay, assuming you didn't order a second optical drive. Even if you did, just place it into an external enclosure, as external RAID enclosures are more expensive. For mounting, solutions exist, though not inexpensive. You could also DIY a mount from an old CD/DVD drive. Just dissasemble it, and use one of the metal plates from the enclosure. Just drill some holes.

Hmm, yeah... hmmm... <Collecting my thoughts> Ya know... There really isn't anything that RAID 0 hasn't sped up for me. From rendering large frames, editing video, direct to disk (DTD) audio recording (even though audiophiles say it's not best), system booting, really everything. I also just about never see scratch usage. If PS is doing it I don't know about it - even under fairly heavy loads. So with a decent amount of RAM unless you working on multiple layers of 20,000 pixels sq., when does this ever actually get used? I'm beginning to think that in this day and age of 16 GB of RAM or more, that we can start safely ignoring Adobe's recommendations to assign the cache to a fast separate partition. I don't have 64-bit PS on any machine so I don't know if that will profile differently or not. So currently I guess someone working in PS would hit the disk cache once a month for a few minutes at most. Is that worth planning for?

Also the way OS X seems to work and the mammoth CPU caches we have today do code calls ever actually cause any disk activity after the initial use? So for a typical single user using one app at a time (even if the others are loaded) to have apps and data on the same partition (or same RAID set in a different partition) is really detrimental to performance? Really? I question this. Maybe it was when L3 caches were less than 512K or something. Is it still? Like I say I question this.

I also question the sanity of using RAID 1 in a SOHO (Small Office Home Office) environment. Mostly if you're a good user your RAID 0 is running at 20% ~ 50% full. Well even if you have a massive 3TB RAID stripe 50% of that can be backed up on a daily or weekly basis to a external drive. Certainly RAID 0 and RAID 1 on the same stripe set defeats ALL of the purpose of both systems! Your RAID 0 is going to crawl (slow!) and the RAID 1 (a hardware level security system) isn't backing anything up at all. Just itself = nothing - when itself IS the vary drives it would otherwise be safeguarding.

There's room for argument in most the assertions I'm making here but I'm doing so in order to come to the point that I think a 2 drive RAID isn't all that much faster and there are only 4 drive bays in a Mac Pro. A two drive RAID is between 101% and 250% faster depending on block allocation size, the files you're writing, the drive's cache size, the area of the platter you're accessing, and etc. etc. And mostly it's hitting in the middle of that at around 160%. That's 60% faster, sure but I want more - enough to actually notice most of the time. A 3-drive goes up to 380% and averages are in the 250% range. That's noticeable! Entirely noticeable!

This supports my final point that if you're going to have 4 drives anyway that 3 of them be involved in a RAID-0 somehow. I just recently learned that it's possible to partition drives first and then add those partitions (of same size) into a RAID 0 so this changes the dynamics of my usual recommendation. And I don't know if you can have these "partition RAIDS" overlap nor what the dynamics of doing so would be. Can we partition all 4 for the 1st 75GB creating a 300GB 4-drive partition and then also create another RAID 0 from just 3 of the drives? Is there CPU overhead in having multiple RAID 0's in a Mac Pro system? There's nearly zero overhead for one RAID 0. Questions.. questions...

Usually given 4-drives in a Mac Pro I recommend 3-drives in one RAID 0 and then partition that at it's speed points - usually the 1st 15% is one partition - the fastest, and the last 15% is another - the slowest, and then the middle 70% partition. The 4th drive is then partitioned for the OS, some storage/resource items, and etc. This is the system without backup. And given the new (for me) information above I dunno if it's the absolute best way any more or not.

I like the external incremental back-up philosophy best myself. It's outside the system so not subject to system-wide electrical faults and failures, it can be turned off when not in use thus prolonging it's life tremendously and consuming less power overall, and if lightning fire or water fries the MacPro system the external unit may survive. The external unit doesn't need to be the same size as all internal drives combined. About half size will usually do. It itself can be a RAID 0 if you have the money or it can be a single 1TB, 1.5TB or one of the new 2TB drives. FW800 is good enough if it's a single drive... that's about 100 MB/s and that's all a single drive can do anyway.

Related to the above assertions, a few months ago I changed my system around so that the OS, apps, and high-speed data were all on the same 3-drive RAID 0 and I used the other drive for image, 3D, video, and audio clip storage to see how the system performed differently. The result so far is that it boots faster, apps load a tiny bit faster, and searching the system is quicker. Video editing, Photo & Image editing, DTD audio recording, music mixing & scoring, and large frame writes from lengthy render sessions show no significant increase or decrease in performance. So it doesn't seem to matter that my high-speed data partition is on the same RAID-0 set as the OS and apps.


.

 

[Mac Husky]

Thx a lot to nanofrog and Tesselator for the very interesting discussion about theory and practice of "speed management". Maybe all this RAIDO is a bit of a speed hype not leading to really impressive increase at all which further more has to be planned very carefully. Summarizing for me personally I am realy thinking about going with a WD VelociRaptor as system and apps disk to start with. This seems be a good investment into more speed so far as system and apps should start noticeable faster. And not putting more data on it, it should not be a big deal to switch to an SSD later on, when prices will come down "a bit". Disk 4 will be a big one (1TB or higher) to backup the whole system via Time Machine, from where an external backup can be made frequently also (FireWire). And than see how to organize data on disk 2+3. I guess with 8GB at all Photoshop has enough RAM to use as "scratch" so far. And starting time of 2-3 seconds from WD VR is quit impressive also.

Planning a SSD later on, it might be an option to use disk 1+2+3 for RAIDO on system, apps and data on WDs Caviar Black or RE3 and backup on disk 4 + extern.
So you wan´t have to buy a WD VR for at least a year or so. On the other hand: for "boots faster, apps load a tiny bit faster, and searching the system is quicker" it doesn´t seem to be a promising way at all.

Buying a WD VelociRaptor and replace it by a SSD in a year or so will probably be the best way to increase speed of system and apps so far.
Later on the WD VR will become a fast data disk in position 2 as well.

 

[Tesselator]

That'll work. And the WD VR is a nice drive! But I hope you didn't misunderstand. I was saying that a 2-drive RAID 0 on average gives you a 60% increase (1.6x averaged across all tasks) and as much as 120% increase at some tasks. A 3-drive RAID 0 will give you about 150% increase (250% of total - or 2.5 times) averaged across all tasks and as much as 280% increase (3.8 times!) at some things. And the higher end "some things" are usually LARGE file or continuous read/write operations right where the higher speed is needed.

It's the planning or "over planning" plus vendor recommendations that I'm not so sure about. It so needed? It's critical? My general and very unscientific tests so far tell me that most of that is just a bunch of anal hooey even tho I've always followed it in the past myself. <shrug>


.

 

[Mac Husky]

That'll work. And the WD VR is a nice drive! But I hope you didn't misunderstand. I was saying that a 2-drive RAID 0 on average gives you a 60% increase (1.6x averaged across all tasks) and as much as 120% increase at some tasks. A 3-drive RAID 0 will give you about 150% increase (250% of total - or 2.5 times) averaged across all tasks and as much as 280% increase (3.8 times!) at some things.

Yes, I got that. My conclusion was: RAIDO with 3 disks or no RAIDO at all.
As I build a totaly new system I may try during the setup.
Problem will be: disk decision should be made before

 

[nanofrog]

Certainly RAID 0 and RAID 1 on the same stripe set defeats ALL of the purpose of both systems! Your RAID 0 is going to crawl (slow!) and the RAID 1 (a hardware level security system) isn't backing anything up at all. Just itself = nothing - when itself IS the vary drives it would otherwise be safeguarding.

Good point. I don't think I really explained this one well enough, as some of the detail is missing. I was thinking mostly in terms of the rebuilds being a nightmare.

There's room for argument in most the assertions I'm making here but I'm doing so in order to come to the point that I think a 2 drive RAID isn't all that much faster and there are only 4 drive bays in a Mac Pro. A two drive RAID is between 101% and 250% faster depending on block allocation size, the files you're writing, the drive's cache size, the area of the platter you're accessing, and etc. etc. And mostly it's hitting in the middle of that at around 160%. That's 60% faster, sure but I want more - enough to actually notice most of the time. A 3-drive goes up to 380% and averages are in the 250% range. That's noticeable! Entirely noticeable!

Here we certainly agree. A 2 drive array isn't going to offer much. Not what would be expected anyway, resulting in dissappointment.

This supports my final point that if you're going to have 4 drives anyway that 3 of them be involved in a RAID-0 somehow. I just recently learned that it's possible to partition drives first and then add those partitions (of same size) into a RAID 0 so this changes the dynamics of my usual recommendation. And I don't know if you can have these "partition RAIDS" overlap nor what the dynamics of doing so would be. Can we partition all 4 for the 1st 75GB creating a 300GB 4-drive partition and then also create another RAID 0 from just 3 of the drives? Is there CPU overhead in having multiple RAID 0's in a Mac Pro system? There's nearly zero overhead for one RAID 0. Questions.. questions...

By making multiple arrays on the same drives, you end up with slower throughput. Lets say you have 3 drives, each with 3 partitions. You end up with 3 RAID0 arrays. Now you start working on something, and requests are made simultaneously for all 3 arrays. Overlap as you call it. You won't get the max throughput of each array, as the drive is attempting to handle the 3 arrays on it at the same time. This slows you down. Now the fact RAID 0 is used, it would make a smaller dent compared to say 5 or 6, but it still occurs. Generally speaking, not ideal, and is best avoided IMO.

Usually given 4-drives in a Mac Pro I recommend 3-drives in one RAID 0 and then partition that at it's speed points - usually the 1st 15% is one partition - the fastest, and the last 15% is another - the slowest, and then the middle 70% partition. The 4th drive is then partitioned for the OS, some storage/resource items, and etc. This is the system without backup. And given the new (for me) information above I dunno if it's the absolute best way any more or not.

For me, it's better to plan from the Min throughput. Worst case planning can get a little more expensive (addtional drives, and perhaps a more expensive RAID card for extra ports), but you know what you've got. If you don't know what you're minimum needs are, experimentation is the best way to find out, and may actually be necessary (situations never seen before = no experience to draw from).

I like the external incremental back-up philosophy best myself. It's outside the system so not subject to system-wide electrical faults and failures, it can be turned off when not in use thus prolonging it's life tremendously and consuming less power overall, and if lightning fire or water fries the MacPro system the external unit may survive. The external unit doesn't need to be the same size as all internal drives combined. About half size will usually do. It itself can be a RAID 0 if you have the money or it can be a single 1TB, 1.5TB or one of the new 2TB drives. FW800 is good enough if it's a single drive... that's about 100 MB/s and that's all a single drive can do anyway.[/QOUTE]
Definitely a better way to go. Size of the backup varies though, and in cases where backups are a mixture of entire system and incrementals, it can get larger than the primary array. An email server is a perfect example. Make a system backup (entire) monthly, and nightly incrementals. Assuming the server handles a large number of accounts, you can run out of drive space. So setting a retention span (say 1 month) helps to mitigate this.

Such a backup plan is up to the user/administrator in a SOHO environment, but is also assisted by the fact it won't be used by 1000's of users. But they still have to decide what they wish to do, and use that to base their decisions on. Hopefully, they won't fill the array that fast, and the backup drives won't have to be huge, or will be filled quickly. But in the end, worst case here is just get another drive. Rather cheap when compared to the rest of the system. Particularly compared to the cost of data recovery.

Related to the above assertions, a few months ago I changed my system around so that the OS, apps, and high-speed data were all on the same 3-drive RAID 0 and I used the other drive for image, 3D, video, and audio clip storage to see how the system performed differently. The result so far is that it boots faster, apps load a tiny bit faster, and searching the system is quicker. Video editing, Photo & Image editing, DTD audio recording, music mixing & scoring, and large frame writes from lengthy render sessions show no significant increase or decrease in performance. So it doesn't seem to matter that my high-speed data partition is on the same RAID-0 set as the OS and apps.

If possible, placing everything on a single array can improve the overall performance of the system. However, redundancy becomes more important IMO, as recovery, even with a proper backup system, can cost quite a bit of time, and money if the work is being done by an IT person (paid with $$$, not beer, or some other tasty beverage for a friend). If it's yourself, a royal PITA. One I prefer to avoid.

Thx a lot to nanofrog and Tesselator for the very interesting discussion about theory and practice of "speed management". Maybe all this RAIDO is a bit of a speed hype not leading to really impressive increase at all which further more has to be planned very carefully. Summarizing for me personally I am realy thinking about going with a WD VelociRaptor as system and apps disk to start with. This seems be a good investment into more speed so far as system and apps should start noticeable faster. And not putting more data on it, it should not be a big deal to switch to an SSD later on, when prices will come down "a bit". Disk 4 will be a big one (1TB or higher) to backup the whole system via Time Machine, from where an external backup can be made frequently also (FireWire). And than see how to organize data on disk 2+3. I guess with 8GB at all Photoshop has enough RAM to use as "scratch" so far. And starting time of 2-3 seconds from WD VR is quit impressive also.

Planning a SSD later on, it might be an option to use disk 1+2+3 for RAIDO on system, apps and data on WDs Caviar Black or RE3 and backup on disk 4 + extern.
So you wan´t have to buy a WD VR for at least a year or so. On the other hand: for "boots faster, apps load a tiny bit faster, and searching the system is quicker" it doesn´t seem to be a promising way at all.

Buying a WD VelociRaptor and replace it by a SSD in a year or so will probably be the best way to increase speed of system and apps so far.
Later on the WD VR will become a fast data disk in position 2 as well.

Use additional drives. If the optical bay is empty, you can have 5 HDD's. You can add a second drive (6 max) to the optical bay, and use a SATA card (preferably with eSATA for future growth; external) for a port, if you have an available slot.

Beyond that, you have to go external. On this note, place the backup drive in a single external enclosure, and use the HDD bay for an additional RAID disk. It's an inexpensive solution, particularly for a MP.

I use a VR, and it's not bad, but I'm somewhat dissappointed with mine. I'm only getting an Avg. throughput of 97.8MB/s, and it should be more like 107MB/s. I get higher Avg. throughput from the RE3s (107MB/s), only with slower Random Access! (I need to contact WD though, as it seems to be firmware related. Everything else, such as SMART and sectors checks out).

Overall though, I don't consider my experience common. Just look at the posts on the forum. It seems many users are quite happy.

It's the planning or "over planning" plus vendor recommendations that I'm not so sure about. It so needed? It's critical? My general and very unscientific tests so far tell me that most of that is just a bunch of anal hooey even tho I've always followed it in the past myself. <shrug>

What reccomendations are you refering to?

The stuff I usually see is pertinant. HDD Compatibility lists, memory (i.e. ranking), motherboard/system,... It's hardware though. I usually shy away from software.

Yes, I got that. My conclusion was: RAIDO with 3 disks or no RAIDO at all.
As I build a totaly new system I may try during the setup.
Problem will be: disk decision should be made before

3 disks or more.
You should know what drives you're going to use first. Research pays off here, especially if you ever switch to a hardware solution. It can be done the other way arround, but can get expensive (RMA's involving restocking fees, and shipping). Not to mention the time spent and resulting headaches.

 

[Tesselator]

By making multiple arrays on the same drives, you end up with slower throughput. Lets say you have 3 drives, each with 3 partitions. You end up with 3 RAID0 arrays. Now you start working on something, and requests are made simultaneously for all 3 arrays. Overlap as you call it. You won't get the max throughput of each array, as the drive is attempting to handle the 3 arrays on it at the same time. This slows you down. Now the fact RAID 0 is used, it would make a smaller dent compared to say 5 or 6, but it still occurs. Generally speaking, not ideal, and is best avoided IMO.

Yeah I kinda thought so. But how about partitions on a single RAID stripe? I mean, I know the dynamics and variables of this but is there a difference in comparison to the partition-first-make-multiple-RAIDs way?

For me, it's better to plan from the Min throughput. Worst case planning can get a little more expensive (addtional drives, and perhaps a more expensive RAID card for extra ports), but you know what you've got. If you don't know what you're minimum needs are, experimentation is the best way to find out, and may actually be necessary (situations never seen before = no experience to draw from).

Yeah, if the 3-part 3-drive RAID 0 isn't going to be enough then for sure you gotta beef up. And yup! Tests are the key.

Such a backup plan is up to the user/administrator in a SOHO environment, but is also assisted by the fact it won't be used by 1000's of users. But they still have to decide what they wish to do, and use that to base their decisions on. Hopefully, they won't fill the array that fast, and the backup drives won't have to be huge, or will be filled quickly. But in the end, worst case here is just get another drive. Rather cheap when compared to the rest of the system. Particularly compared to the cost of data recovery.

Yep, SOHO as defined in Japan is usually a single guy working at home, But it can be and often is 2 to 4 people in a house or an apartment used as an office. At 5 or more people they start calling it a company - even though both actually are.

If possible, placing everything on a single array can improve the overall performance of the system. However, redundancy becomes more important IMO, as recovery, even with a proper backup system, can cost quite a bit of time, and money if the work is being done by an IT person (paid with $$$, not beer, or some other tasty beverage for a friend). If it's yourself, a royal PITA. One I prefer to avoid.

Yeah, I was considering everything as DIY - like the person asking the original question. But back-up can still be just a single drive. 3hrs. for a 1TB disk and 4.5 to 5 hrs. for a 2 TB drive. That's to write the entire space of the destination drive via simple drag-n-drop. Some backup software accelerates the process via fast compression and data reformatting. Assuming the RAID is 2 or 3 TB that should do. Or did I miss your meaning?

Use additional drives. If the optical bay is empty, you can have 5 HDD's. You can add a second drive (6 max) to the optical bay, and use a SATA card (preferably with eSATA for future growth; external) for a port, if you have an available slot.

That's actually a pretty good tip! What would be awesome is if we could run the power line to an external switch so we could leave it in the OFF position most of the time.

Beyond that, you have to go external. On this note, place the backup drive in a single external enclosure, and use the HDD bay for an additional RAID disk. It's an inexpensive solution, particularly for a MP.

I use a VR, and it's not bad, but I'm somewhat dissappointed with mine. I'm only getting an Avg. throughput of 97.8MB/s, and it should be more like 107MB/s. I get higher Avg. throughput from the RE3s (107MB/s), only with slower Random Access! (I need to contact WD though, as it seems to be firmware related. Everything else, such as SMART and sectors checks out).

Overall though, I don't consider my experience common. Just look at the posts on the forum. It seems many users are quite happy.

97 ~ 107 MB/s is pretty good though! I mean compared to the 50 ~ 60 MB/s we were getting out of the 500GB drives just a few years ago.

What reccomendations are you refering to?

The stuff I usually see is pertinant. HDD Compatibility lists, memory (i.e. ranking), motherboard/system,... It's hardware though. I usually shy away from software.

You're right about that. I was talking more about Adobe's recommendations for the scratch and many audio, photo, and video editing software vendors will recommend various methods that they think will improve performance. I don't mean the hard specs - those are requirements. Adobe is a good example here - everyone knows what they say about where to put their scratch. So far most of my tests show that this is a bunch of hooey to a large degree. Scratch on a partition by itself, on a dedicated drive, on the application drive... it doesn't seem to matter at all. The performance is whatever the drive can deliver. So if your applications RAID 0 partition is the same speed as another unused RAID 0 partition in your system you will not be able to tell any speed difference when it goes to the disk cache. The drive speed for that kind of data - whatever we're talking about - is what you get. Where and what else shares it (in a single user desk-top system) just doesn't seem to matter at all. Of course it's common sense that if you're making two programs work in the same area at the same time then that'll suck... I'm going on the belief or proposition, that this almost never actually happens or if it does the user can change his bad behavior!

.

 

[Mac Husky]

This thread is becoming more and more interesting and I am trying hard to follow you both in every point as english is not my native language

So I hope you don´t mind, if I might ask some stupid questions, that are already answered - I might have overlooked something due to all that information stuff

Use additional drives. If the optical bay is empty, you can have 5 HDD's. You can add a second drive (6 max) to the optical bay, and use a SATA card (preferably with eSATA for future growth; external) for a port, if you have an available slot.

My second optical slot is free. If it turns out to be necessary, I hope you don´t mind me to contact you again for further procedure.

As external backup disk I am planning a WD Studio Edition, which has FireWire 800 and eSATA connectivity as well. Would it be useful, to connect it via eSATA instead of FireWire and if yes: HOW in detail?!

Here we certainly agree. A 2 drive array isn't going to offer much. Not what would be expected anyway, resulting in dissappointment.

I was talking more about Adobe's recommendations for the scratch and many audio, photo, and video editing software vendors will recommend various methods that they think will improve performance. I don't mean the hard specs - those are requirements. Adobe is a good example here - everyone knows what they say about where to put their scratch. So far most of my tests show that this is a bunch of hooey to a large degree. Scratch on a partition by itself, on a dedicated drive, on the application drive... it doesn't seem to matter at all. The performance is whatever the drive can deliver. So if your applications RAID 0 partition is the same speed as another unused RAID 0 partition in your system you will not be able to tell any speed difference when it goes to the disk cache. The drive speed for that kind of data - whatever we're talking about - is what you get. Where and what else shares it (in a single user desk-top system) just doesn't seem to matter at all.

OK. You both say no RAIDO under using 3 disks at minimum. And - when I got the last point right - you don´t see any benefit of a RAIDO for Adobe data e.g. Photoshop pictures at all?! Right?!

The only apps, I would like to have benefit from RAIDO are Adobe Photoshop, Adobe InDesign/Illustrator and in the future Final Cut Express. I don´t need no RAIDO for any other app I am using right now. My main work is on RAWs using Nikon Capture NX, Aperture or Lightroom stuff and Photoshop. So getting you right, there will be no significant benefit on this kind of data by building a RAID0. Just for the apps itself there would be a speed increase putting the software on RAIDO by minimum of 3 disks???

You should know what drives you're going to use first. Research pays off here, especially if you ever switch to a hardware solution.

Right. That´s why I am trying to understand all the stuff you mentioned here prior to my decision what to buy Trying to figure out what is useful and what isn´t at all.

 

[Tesselator]

FinalCut likes RAID... Photoshop, Indesign, and Illustrator don't care. However It might depend with Indesign. I wrote the LightWave 3D manual and with 700 pages loaded almost all with multiple hirez graphics it can get kinda slow. And it goes to the disk to load images not already in it's cache or that have fallen out of the cache and are paged to again. So you're doing more than say ~100 pages with lots of graphics then a RAID would probably speed it up.


Also we're not saying 2-drive is no benefit. Just usually not as noticeable a difference as 3-drive or even 4-drive. Maybe this will help:

Courtesy Of Tom's Hardware
​

This is a pretty good RAID controller on Windoze with a little slower 300GB drives from early 2007. For Mac Pro's software RAID it doesn't scale quite so evenly. Also the more drives you have in the RAID stripe the more combined disk cache you have and that reads and writes at the full interface speed of usually 3 gigabytes per second. if that's 8MB then a 3 drive RAID is 24MB and etc. While 8MB isn't even a single RAW 10MP photograph 24MB is two of them.

 

[nanofrog]

Yeah I kinda thought so. But how about partitions on a single RAID stripe? I mean, I know the dynamics and variables of this but is there a difference in comparison to the partition-first-make-multiple-RAIDs way?

It's on the same drives, so accessing data on one array affects the througput on any other, as the total bandwidth is fixed (number of drives and controller ports used). It gets worse if the hardware involves a PM (Port Multiplier). Real results depends on specifics though. Again, testing specifics (usage) is the best way to be sure.

Yeah, I was considering everything as DIY - like the person asking the original question. But back-up can still be just a single drive. 3hrs. for a 1TB disk and 4.5 to 5 hrs. for a 2 TB drive. That's to write the entire space of the destination drive via simple drag-n-drop. Some backup software accelerates the process via fast compression and data reformatting. Assuming the RAID is 2 or 3 TB that should do. Or did I miss your meaning?

I was referring to the redundancy of the array, depending on the type used, not the details of the backup. That is, RAID 0 has no redundancy (fail over) at all, while others such as 1,10,5,6,50,60 do. But each is different from one another (number of failed drives before the array is gone).

Given the lack of redundancy, I stay away from RAID 0. I just don't trust it, and am willing to put the extra cash into a controller and drives. Backup solutions should also tailored. It can be another identical array, to a single drive, managed with a decent backup software application.

That's actually a pretty good tip! What would be awesome is if we could run the power line to an external switch so we could leave it in the OFF position most of the time.

Actually, you don't need a switch. The drive will automatically spin down if not in use (part of SMART functionality). Management of this function is in the OS, usually in the Power Management control section. The controller board on the drive stays active, so if a read/write is sent, it spins up, then performs the request.

It's different for hardware based RAID solutions though, as the OS can't manage it. The card has to manage the drives, as they're operated differently, and it's usually an added feature (MAID). Unfortunately, it isn't always there, so careful attention needs to be paid to the specs of the card. It's not as important if enterprise drives are used, as they're built for it, but the lack of such a feature can kill consumer units.

97 ~ 107 MB/s is pretty good though! I mean compared to the 50 ~ 60 MB/s we were getting out of the 500GB drives just a few years ago.

So I'm greedy.

You're right about that. I was talking more about Adobe's recommendations for the scratch and many audio, photo, and video editing software vendors will recommend various methods that they think will improve performance. I don't mean the hard specs - those are requirements. Adobe is a good example here - everyone knows what they say about where to put their scratch. So far most of my tests show that this is a bunch of hooey to a large degree. Scratch on a partition by itself, on a dedicated drive, on the application drive... it doesn't seem to matter at all. The performance is whatever the drive can deliver. So if your applications RAID 0 partition is the same speed as another unused RAID 0 partition in your system you will not be able to tell any speed difference when it goes to the disk cache. The drive speed for that kind of data - whatever we're talking about - is what you get. Where and what else shares it (in a single user desk-top system) just doesn't seem to matter at all. Of course it's common sense that if you're making two programs work in the same area at the same time then that'll suck... I'm going on the belief or proposition, that this almost never actually happens or if it does the user can change his bad behavior!
.

Ahh... Got it. I've not paid a lot of attention, and presumed there was something to it, as the advice is from the developer. I never investigated it further.

 

[nanofrog]

This thread is becoming more and more interesting and I am trying hard to follow you both in every point as english is not my native language

So I hope you don´t mind, if I might ask some stupid questions, that are already answered - I might have overlooked something due to all that information stuff

Not a problem.

My second optical slot is free. If it turns out to be necessary, I hope you don´t mind me to contact you again for further procedure.

This will make things easier, save you some time, and save on cost.

You're welcome to send a PM.

As external backup disk I am planning a WD Studio Edition, which has FireWire 800 and eSATA connectivity as well. Would it be useful, to connect it via eSATA instead of FireWire and if yes: HOW in detail?!

Either interface will transmit the data to the drive, but eSATA is faster. If you have time, it may not matter to you. However, if you will be backing up large files (large capacity overall), the eSATA port may be a better alternative, as it can get the information written faster. This will be important once the drives age, and are more prone to failure. (When a failure occurs during a backup procedure).

HOW is quite easy. You obtain an eSATA card (that works with a Mac), and place it in an empty PCIe slot. Install the drivers, and you will be able to see a drive attached to it.

OK. You both say no RAIDO under using 3 disks at minimum. And - when I got the last point right - you don´t see any benefit of a RAIDO for Adobe data e.g. Photoshop pictures at all?! Right?!

The only apps, I would like to have benefit from RAIDO are Adobe Photoshop, Adobe InDesign/Illustrator and in the future Final Cut Express. I don´t need no RAIDO for any other app I am using right now. My main work is on RAWs using Nikon Capture NX, Aperture or Lightroom stuff and Photoshop. So getting you right, there will be no significant benefit on this kind of data by building a RAID0. Just for the apps itself there would be a speed increase putting the software on RAIDO by minimum of 3 disks???

You'd benefit from the RAID, but you wouldn't need to make multiple arrays on the same drives. Just one would be sufficient IMO. The more drives you have, the faster it can go, but at the increased risk of a drive failure. So don't go crazy. 3 drives would be fine, perhaps even 4. But I would HIGHLY recommend enterprise drives, not consumer models. Your data is safer on them (increased Unrecoverable Bit Error).

From what I'm understanding from Tesselator, just place everything on the array. No need to try and separate everything. OS is up to you, if you wish to place it on a separate drive. If you do, at least the computer will still boot, if the array ever goes down. It makes it easier to rebuild from backups at least.

Right. That´s why I am trying to understand all the stuff you mentioned here prior to my decision what to buy Trying to figure out what is useful and what isn´t at all.

No problem. At least you're trying to learn.

You might also want to search Wiki, as there's a fair bit of useful information regarding RAID.

 

[Mac Husky]

Thanks again, guys. I appreciate it.

Summing up the important things for me:

- RAID0 is OK with 2 but much better with 3 or more drives
- RAID0 does not seem to bring a big benefit for Photoshop data as long as you don´t have xxx files open at the same time
- RADI0 does have benefit for apps itself
- RAID0 will have additional benefit when using Final Cut Express later on
- RAID0 should be proceded with RAID class drives

For me this would lead to the following options:

a) drive 1+2+3 in a RAID0 with all apps and data (maybe OS as well) fully backup on drive 4 + extern
b) no RAID0 at all as long as I don´t suffer from having none

Way (a) would mean: port 1+2+3 with WD RE3, port 4 with a big WD Caviar Black
Way (b) would mean: port 1 with WD VR (system, apps - maybe SSD in a year?!), port 2 + 3 (+ 4) WD Caviar Black (of different sizes)

Going way (a) I will think about using the optical port again, but I am not shure, whether a BlueRay might be useful/possible later on for Final Cut Express data doing HD video.