Questions on setting up new MP (RAID, drive selection and other noob questions)

[MCHR]

After doing quite a bit of overall searching, I thought I'd ask the masses about setting up a new MP for my workflow. JulianBoolean's thread has good information, but it's gotten pretty technical so I'm asking in terms I relate to. . . sorry


Anyway. My main workflow is in 2D imaging. Photoshop, Corel Painter, etc. I may open Rhino for a 3D app, or play around in Maya at some later date to broaden my marketability

I have been doing a fair amount of freelance, so reliability is a concern. Initially, I'd like to know about mirroring for file security.

My current setup is a PowerMac G5 with a 74GB Raptor boot and 160GB Maxtor second storage drive (internal). I have an external LaCie for archiving. No issues with any of that hardware. Backup and archiving has been manual to this point. No Time Machine, etc.

I just received a 3.2 quad MP. 6GB ram. I am looking to add an OCZ Vertex 2 60GB drive in the main boot drive for quick access. I bought two Western Digital 500GB Black caviars that I'd like to set up in a mirror configuration (is that sensible?) Possibly another SSD as a scratch disc for graphics and Photoshop work.

My questions are : Is there a better drive than the Caviars for this use? Is an RE3 or RE4 any better for this application?

Also, would any RAID make sense for me? I have never used RAID in any computers to date.

Thanks for any advice.

 

[Hellhammer]

RE3 and RE4 are better because they are designed for enterprise use and thus have better reliability. Caviar Blacks should be fine though. If you want to use them in mirror mode, you need to use RAID 1.

 

[MCHR]

RE3 and RE4 are better because they are designed for enterprise use and thus have better reliability. Caviar Blacks should be fine though. If you want to use them in mirror mode, you need to use RAID 1.

So, as I understand it, would I format and load apps 'normally' on the SSD boot drive? Then, do I initialize and format the two Caviars in RAID 1 afterward?

I assume that I would use one of those drives 'active' and the second would duplicate that data in a transparent fashion (?)

 

[Hellhammer]

So, as I understand it, would I format and load apps 'normally' on the SSD boot drive? Then, do I initialize and format the two Caviars in RAID 1 afterward?

Yes, you install the OS X to the SSD and use it normally. You may want to move the Home folder to the HD though

I assume that I would use one of those drives 'active' and the second would duplicate that data in a transparent fashion (?)

If you put those drives in RAID 1, you will see an array of 500GB that you can use. Then if you put something to that array, it will automatically write it to both HDs.

 

[MCHR]

Yes, you install the OS X to the SSD and use it normally. You may want to move the Home folder to the HD though



If you put those drives in RAID 1, you will see an array of 500GB that you can use. Then if you put something to that array, it will automatically write it to both HDs.

Great. That clears things up tremendously. Last question is whether I need a RAID card for that configuration. Would you favor the RE3 or the RE4 drives? It seems their specs are close except for the buffer size.

Oh yes, and the RE3's have a 5 yr warranty, RE4's have a 3 yr.

 

[Hellhammer]

Last question is whether I need a RAID card for that configuration.

Nah, you can use software RAID for RAID 0 and 1. You need a RAID card for RAIDs like RAID 5 for example.

Would you favor the RE3 or the RE4 drives? It seems their specs are close except for the buffer size.

As you're running RAID 1, I wouldn't. If it was RAID 0 or other RAID where reliability is bigger concern, then I would.

 

[MCHR]

Nah, you can use software RAID for RAID 0 and 1. You need a RAID card for RAIDs like RAID 5 for example.



As you're running RAID 1, I wouldn't. If it was RAID 0 or other RAID where reliability is bigger concern, then I would.

So, since I'm only using RAID to duplicate (mirror) my files when working, the Black Caviars would be a good choice?

I'm only asking since the WD RE3 appears to be validated for longevity. The Caviars look speedier, so it may be a tradeoff.

 

[Hellhammer]

So, since I'm only using RAID to duplicate (mirror) my files when working, the Black Caviars would be a good choice?

Yeah, they should be fine. For RAID 0 and 1, I wouldn't bother with special drives but if you are using hardware RAID, then it's better to use drives that are designed for RAIDing.

I'm only asking since the WD RE3 appears to be validated for longevity. The Caviars look speedier, so it may be a tradeoff.

All hard drives will eventually die. Usually they serve you for years, even the normal drives. Besides, Caviar Black is prosumer drive, it's not that mainstream crap

 

[nanofrog]

I just received a 3.2 quad MP. 6GB ram. I am looking to add an OCZ Vertex 2 60GB drive in the main boot drive for quick access. I bought two Western Digital 500GB Black caviars that I'd like to set up in a mirror configuration (is that sensible?) Possibly another SSD as a scratch disc for graphics and Photoshop work.

My questions are : Is there a better drive than the Caviars for this use? Is an RE3 or RE4 any better for this application?

Also, would any RAID make sense for me? I have never used RAID in any computers to date.

Keep in mind, that a mirror created under Disk Utility (OS X) will operate at the same speed as a single disk.

Now consider you really want the primary location and scratch location to run at or near the same speed (i.e. so one disk/array isn't a bottleneck for your workflow). That is, if you have a fast scratch, and a single disk throughput for the primary data, the system will be less responsive as it waits for writes to the primary location to be completed.

The way to fix this, is either use a single disk for scratch (mechanical, not an SSD), or put the primary data on a faster array, such as a level 10 (1 + 0, where you create a pair of mirrors, then stripe the two together).

As per disks, the Caviar Blacks are good disks, but the REx versions are better.

BTW, a mirror is meant for availability = operational as much as possible (think 24/7 operation). It is NOT A BACKUP, as if you make a mistake, it gets duplicated automatically (i.e. delete a file you needed to keep).

So single disk operation or RAID, you need a sufficient backup system for your needs.

 

[Honumaui]

Now consider you really want the primary location and scratch location to run at or near the same speed (i.e. so one disk/array isn't a bottleneck for your workflow). That is, if you have a fast scratch, and a single disk throughput for the primary data, the system will be less responsive as it waits for writes to the primary location to be completed.

just to clarify one thought using PS
for PS the above wont be the case ? might be for other scratch but at least for PS the scratch should always be on the faster HDD no matter what as long as that is not the same HDD as your files you are working on
but the scratch in PS is purely their when memory runs out since your files are not writing out ?

 

[2contagious]

BTW, a mirror is meant for availability = operational as much as possible (think 24/7 operation). It is NOT A BACKUP, as if you make a mistake, it gets duplicated automatically (i.e. delete a file you needed to keep).

Is mirroring a backup solution in terms of when 1 drive fails though? If data is written to both and 1 of the drives fails, the data is still on the other drive, isn't it?

Also, what are the speed improvements like when using a RAID0 configuration on two 1GB Caviar Blacks?

 

[nanofrog]

just to clarify one thought using PS
for PS the above wont be the case ? might be for other scratch but at least for PS the scratch should always be on the faster HDD no matter what as long as that is not the same HDD as your files you are working on
but the scratch in PS is purely their when memory runs out since your files are not writing out ?

I understand what you're getting at, but I keep thinking of the slowest link in the entire chain.

So in the case where a single HDD is used for data, you don't want to get too crazy about scratch speed, as even though the processing is sped up, the single disk is still the limitation for overall workflow (processor has to wait for the data to be written, particularly as the RAM is filled).

Is mirroring a backup solution in terms of when 1 drive fails though? If data is written to both and 1 of the drives fails, the data is still on the other drive, isn't it?

Also, what are the speed improvements like when using a RAID0 configuration on two 1GB Caviar Blacks?

A mirror is meant for availability (i.e. 24/7 operation). A mirror allows the system to keep running if one of the drives fails (i.e. place the OS and applications on a mirror set).

As a backup, it's not actually meant for that, as if you make a mistake, it's automatically duplicated to the other disk in the set. So if you delete a file you didn't mean to, it's gone without a proper backup system in place.

As per a stripe set (RAID 0):

Performance:

• n members * performance of a single disk. So if you have 2x disks that are capable of 100MB/s each, the stripe set of the pair produces 200MB/s. The downside is, if one disk fails, all the data is gone.

Reliability:

• 1/n * reliability of a single disk So in a 2 disk set, the reliability = 1/2 of a single disk.

 

[2contagious]

As per a stripe set (RAID 0):

Performance:

• n members * performance of a single disk. So if you have 2x disks that are capable of 100MB/s each, the stripe set of the pair produces 200MB/s. The downside is, if one disk fails, all the data is gone.

Reliability:

• 1/n * reliability of a single disk So in a 2 disk set, the reliability = 1/2 of a single disk.

Is there any situation (for graphic design / photography / film use) that might benefit from having a RAID 0 setup with two HDDs (I have two 1TB drives) if I already have a single SSD (OWC 120GB) as a boot drive?

 

[MCHR]

The way to fix this, is either use a single disk for scratch (mechanical, not an SSD), or put the primary data on a faster array, such as a level 10 (1 + 0, where you create a pair of mirrors, then stripe the two together).

As per disks, the Caviar Blacks are good disks, but the REx versions are better.

Is there a reason an SSD wouldn't be a good choice for a scratch disc? I'd assume that the quicker access, the better. But your point may have more to do with SSDs being better are 'read' functions instead of swapping and 'write / read'. Is this correct?

 

[nanofrog]

Is there any situation (for graphic design / photography / film use) that might benefit from having a RAID 0 setup with two HDDs (I have two 1TB drives) if I already have a single SSD (OWC 120GB) as a boot drive?

You can. But whether or not it's a good idea, depends on your usage and budget. That is, for a professional (data really is critical, as is their time), using a stripe set for primary data is a bad idea (takes time to recover, and that includes re-performing work that occured between the last backup and the failure). They're better served using a RAID level that includes redundancy as well as performance (reduces the amount of time they have to put in recovery themselves, and the data is safer as well).

In the event that a user doesn't have a proper backup in place, the data is just gone. Data recovery services may be able to help, but it's extremely expensive (~$2k+ per disk last I priced it ).

As per backups, you need them. RAID itself is no substitution, which means RAID or single disk, you need to have a proper backup for your situation. Unless you're fine with total data loss. Assuming this is the case, the argument for backup or redundancy for specific usage doesn't apply.

For an enthusiast/hobbyist (those that can afford the time, and usually have very limited budgets), the trade-off of your time may be an acceptable compromise to use a stripe set for primary data. But you must run a proper backup if you want any chance of recovering lost data. Otherwise you're just screwed.

An instance where it can be quite helpful in terms of performance and low cost (any user), is as dedicated scratch space for applications such as Photoshop (no need to back this up either, as it's all temp data).

Is there a reason an SSD wouldn't be a good choice for a scratch disc? I'd assume that the quicker access, the better. But your point may have more to do with SSDs being better are 'read' functions instead of swapping and 'write / read'. Is this correct?

You can use SSD's as scratch space if you wish, but you need to realize that SSD's aren't meant for high write conditions. The reason is the MLC Flash used is only good for 10,000 writes (some less) per the Flash manufacturer. Wear leveling is meant to improve lifespan by rotating writes across cells before a cell is re-written again (if it's available). So expensive SSD's aren't a good idea (this has been discussed in detail in other threads). SLC is capable of many more writes (100,000 write limit per cell), but is horribly expensive. There's a new form of MLC called eMLC created by Micron, but it's still more expensive than the usual MLC available (and is why it's the most commonly used form right now). Always the financial aspect....

Now with the 40GB model available from OWC for $100, it's more of a possibility if the user is willing to use a 1 - 1.5 year MTBR (Mean Time Between Replacement = toss it and put in a new one). Personally for a pro, the compromise means $100 per year, and should be acceptable as they're earning a living with the system.

Technically, the compromise listed above has been there, but given the costs of other SSD's, it's been too prohibitive for most, as virtually no one has an unlimited budget. So other solutions were a better fit within the confines of their budgets (i.e. SSD for an OS/applications disk, and mechanical in some configuration for other areas; maybe RAID was viable, maybe not).

This is a recent change of events that can open up new possibilities that didn't really exist just a month or so ago (prior to this drive shipping, as the inexpensive SSD's <$100 price point> were about the same speed as a single mechanical drive, and just not worth it).

 

[Honumaui]

I understand what you're getting at, but I keep thinking of the slowest link in the entire chain.

So in the case where a single HDD is used for data, you don't want to get too crazy about scratch speed, as even though the processing is sped up, the single disk is still the limitation for overall workflow (processor has to wait for the data to be written, particularly as the RAM is filled).

to expand I think we are both correct ? it really depends on many things

so yes you can get crazy about scratch speed even if a single HDD it is all based on use ?

I can read and write a lot of files quickly tonight I am only doing 36 files in PS complete retouch though
so the opening and closing of files is important but in reality the size about 150 megs for them I am thinking tops will only be a second or so difference total per file so lets call it 2 seconds thats a minute in a 2 hour job
but the scratch itself can make or break it depending on my memory situaion of course
my usual day is about 100 PS files and 1000 raw ?

but all the time in between is where memory and scratch come in ?


so in PS the two of writing and reading files is depending on the disc speed and yes the faster the better depending on file size ? if you are not above 100 or so megs ? no biggy

up to about 300 megs a single drive might be a few seconds to open ? lets say 5 seconds
a raid 1 second
if you only open 10 files but work on each file 30 minutes that is only 50 seconds saved but that scratch might save you 10 minutes per file !

if you worked on 100 files then that savings of 500 seconds starts to add up

the scratch in PS is second to memory !! memory is king

the write proccess as been discussed is CPU limited
and we only open and close once per file but our scratch depending on what we do might be hit 30 times or more while working ?

so yes its important but I treat them seperate depending on the needs of the user
cause as you see if you open and close only 20 images and they are under that 100-150 or so size but you work on them for a while ? then the HDD speed is not that critical

if you did 300 files and were running batches then both are very critical maybe ?

300 files and doing a sharpen then opening and closing is more critical

see what I mean by they are totally seperate and why I would always ask about efficiency and file size and scratch use etc.. how many and so on

 

[Honumaui]

also PS is its own beast as is raw and doing video ? which I know very little about video ? I play with it but thats it
but I know its important to pull in HD stuff and the stuff I play with mostly 5 monute things its nice to have the speed for HD

 

[nanofrog]

to expand I think we are both correct ? it really depends on many things

My position is based on the assumption that the system has sufficient memory for the user's typical usage under Photoshop. Not those that are trying to get away with the memory capacity that shipped with the system (capacity of the base system, which is dismal).

So if it comes down to say $200 in upgrades, go for RAM, rather than a 2 disk stripe set for scratch space.

But if the RAM is sufficient, then don't get carried away (i.e. 1GB/s for scratch space, which is what 4x SSD's could potentially produce vs. single disk for primary data). The single disk writes will choke the life out of workflow. The time savings you listed wouldn't make any difference in such a condition.

 

[Honumaui]

that makes sense
and yes do the ram for sure

since PS will create scratch the second its opened this is something that we cant turn off I think a small SSD can be handy from my extended testing I have been doing daily in real workflow
but this is based on a money making machine otherwise short stroked raids have worked before

its a balance thing example the OP with a few HDD can ask if a single 40 gig for $100 is going to be worth it ? it might depending on finance and how many etc..
I would never say do 4 in a raid 0 ? over kill lucky the magic number seems to be about two anyway on return so then it gets into how big a scratch do you need

 

[MCHR]

Sounds like a small capacity SSD may make sense as a scratch disc if workflow warrants it. I had been considering a small 30 or 40GB SSD anyway. If course ram comes first.

 

[Honumaui]

Sounds like a small capacity SSD may make sense as a scratch disc if workflow warrants it. I had been considering a small 30 or 40GB SSD anyway. If course ram comes first.

ditto ? two in raid 0 might be better ? $225 total ? have not tested smaller single 40 as scratch yet ? but going to soon

just know you could kill it like mentioned ? but I still say warranty and they should not be that fragile we shall see

the other thought would be how are you doing first ! meaning look up your efficiency and your scratch first with your files using the info palette or lower left info in PS files ?

and a SSD for me would come after I am at 24 gigs memory if I had 16 I would throw that money into ram if I was at 24 I would see where I am at and it if its a once in a while thing I would try the single SSD just for that initial hit kinda thing ?
again realize you are entering something that is a who knows ?
just make sure you assign a second disc after the SSD ?

other things you can do is build some purge commands into actions ? you loose your history states etc. but gain back memory ? its a trade off as you go along but it helps with some things
so action first create snapshot of pre state then do action then purge at end ? at least worth checking into this more

 

[nanofrog]

its a balance thing example the OP with a few HDD can ask if a single 40 gig for $100 is going to be worth it ? it might depending on finance and how many etc..
I would never say do 4 in a raid 0 ? over kill lucky the magic number seems to be about two anyway on return so then it gets into how big a scratch do you need

The advent of the 40GB OWC model at the magic $100 price point has become a game changer.

You'll only need one (single SSD @ ~175MB/s vs. 2x mechanical set short stroked for ~ $170 or so for a pair of disks capable of producing the same throughput). It even works out cheaper in terms of a 1 year MTBR vs. 3yr MTBR for the mechanicals ($300 vs. $340 over 3 years).

As new mechanical disks show, this could shift slightly in favor of mechanical again (slight shift one way or the other, depending on which disk tech just released new gear). Such as new mechanical disks with yet another improvement in platter density vs. existing SSD's (target 40GB OWC in this case, but others should follow, and hopefully get faster without pushing the price back up). But SSD has become an option for this particular usage with small, inexpensive models (capacity's not the major issue, performance is).

ditto ? two in raid 0 might be better ? $225 total ? have not tested smaller single 40 as scratch yet ? but going to soon

Maybe. The ICH's bandwidth could be an issue with multiple SSD's attached (for the existing models, it would be possible; 350MB/s for the stripe set, leaves ~310MB/s for the OS/applications disk). A SATA 3.0Gb/s port actually tops out at ~ 270 - 275MB/s anyway (sustained throughputs, not burst).

It's just something to keep in mind, and the specific models need to be checked to see if they add up to more than the ICH limit.

just know you could kill it like mentioned ? but I still say warranty and they should not be that fragile we shall see

Quite true. Anyone who attempts this is a guinea pig ATM, as there's no real world data otherwise to base anything on. But it should last a year. If not, send it back. Simple.