SAS, SATA, Mac Pro RAID, Massive Storage Implications

[mgsarch]

Hi,

I'm sorry if this has been discussed, it's my first real post and although I found info about all these things everything I find is scattered or littered with misinformation. I'm hoping some people that have experience and knowledge with this might be willing to reply and shed some light for me.

I have 4 drives in Mac Pro:

750GB Seagate 7200.11 32MB Cache SATAII (2 days old)
500GB Western Digital SE16 16MB Cache SATAII (1 year old)
500GB Western Digital SE16 16MB Cache SATAII (1 year old)
320GB Western Digital SE16 (Apple Installed, I formatted) (1 week old)

I'm a designer- 3D, Graphic, Video/Animation, Web, Architecture:

Relevant because that's how I use data, massive numbers of files, versioning, etc. Tons of very large files as well.

What I'm trying to achieve:
Create the fastest, safest, easiest to manage storage scheme. I'm guessing that RAID 0+1 is my best best.

Question:
Do I order the Mac Pro RAID card? Are there alternatives that are as good or better? Do I need new drives? I'm kicking myself for not having ordered the server grade drives but in the worst case I can turn these into externals.

Considerations Based On My Use:

1. Transferring data between volumes is driving me insane. I find myself needing to reallocate the drives on a regular basis as space becomes scarce. The speed of the transfer is ok (SATA2, 300MB/s; I get about 50-60mb/s I estimate)but I would love for it to be faster.

2. This scenario presents a nightmare if an organization scheme is not followed exactly at all times. Because similar data has to be spread across volumes there is no reasonable way of maintaining order (aside from diligent use of Version Cue/Subversion or <ghasp> a pen and paper catalog; neither is human-proof).

Anyone who's used illustrator a great deal can probably empathize on how awful it is to sift through (drawing1.ai, drawing2.ai, drawing3.ai, drawing4.ai, drawing5.ai .... drawing137.ai) This exponentially worse when you have multiple volumes to deal with. I don't like dedicating volumes to content because then when one fills up it's another problem of finding somewhere to move stuff.

3. Additionally, there's no nice way to use time machine or spotlight on this kind of setup. They just don't seem to like it and spotlight often hangs while indexing or just "forgets" certain locations. It also tends to find Font Suitcases and go nuts with them. Yes, I could private these folders, but again, that just impedes the organization functionality. RAID redundancy is attractive but mirroring would reduce me to less space than I want. Striping+Parity Checking sounds really great though! I could keep a carbon copy of the OS on an external just in case of a disaster where I couldn't boot.

4. WTF is wrong with spotlight. It breaks every Mac I've owned. I'm sure it's my files but .... wtf. The preferences are so lean and using mdutil is just as bad. Is there another brute force file finding tool that is better, more cohesive, than easyfind?

Any help is very much appreciated. Thank you for reading!

 

[kiomon]

I am a video editior and finisher, so I know about all the versions and files scattered. This will depend a lot on how much money you have to spend on this endeavor.

To run a RAID efficiently you need to have same size and type of drives. You have 4 drives now, if you were dying to keep them. Get the max connect kit and you can put them in your mac pro dvd bays. Then I would add a caldigit raid card (www.caldigit.com) $550 and then get 4 1TB Drives $1000 ($250 each), and run the 4 internal as RAID 5, and you will net with about 2.7GB or so of useable space, and it will be wickedly fast. And then you still have those other 4 drives as JBOD. I would reco 1 for system and apps, and the other drives for "library" elements, things you will use over and over again in jobs (photos, stock projects, etc) Maybe a drive for sound/ music, 1 for photos/ clip, 1 for stock videos. When you work on a specific project, it works off the RAID. With hard drives so cheap now you might as well keep them, as you can't sell them for much.

Hope that helps. If you can't swing the $1700 or so the above costs, you can buy the RAID card and chance the drives, but it will default to the smallest size drive, and you will net with a very small RAID.

 

[mgsarch]

Well, I think I have to do it!

Which CalDigit card are you using? I'm guessing I would go for the 4port PCI-X FASTA.

Why go with CalDigit instead of Apple, it's a little bit cheaper but the price difference isn't enough to make me not consider the Mac Pro Raid.

How necessary is it to get enterprise class drives? I think I remember that the Seagate ES.2 and the 7200.11 are essentially the same drive but the ES.2 will allow RAID error checking to kick in while the 7200.11 is a desktop drive?

What is the deal with SATA and SAS? I think of SAS as being SCSI but I think what I'm seeing is a SAS SATA? Do I need special drives for that?

Is there a good resource I can turn to? I want to order this ASAP.

Is migration going to be a nightmare? sorry for the disjointed note.

 

[kittiyut]

Before ordering the APPLE RAID card, read my post: https://forums.macrumors.com/threads/456854/

 

[Siron]

Sorry if I underestimate your issue (I'm no video editor) but have you considered a DROBO where you can have a 4TB set up that can be expanded (presumably as larger drives become available). I know its slow (USB 2.0) but has the ability for expansion.
Alan

 

[Cryptic1911]

I just setup my mac pro with 3x500gb sataII drives in a raid0 (software) and 2x750gb raid0 eSata and I was getting 8.5-9gb a minute transfer speeds (timed 170gb of it in 19.5mins) once time machine kicked in and stopped doing its initial index or whatever it does. it took quite a while for it to start moving any serious data, but that was the first initial backup of almost 350gb. I've got around 80k files on that raid array of mixed sizes, so it evens out

raid0 isn't recommended because it has no fault tolerance,but it sure is fast. with time machine working for me, the lack of fault tolerance isn't an issue... what was happening to your time machine?

i'm using seagate 7200.10 500gb drives, samsung f1 750gb, OWC eSata dual enclosure, and a newertech eSata extender

 

[Mac_Max]

SAS is a SCSI implementation that also supports connecting SATA drives.

You'll never hit the full speed of SATA on a single drive with today's drives. The best you'll do is about 80-110MB/s under optimal conditions with a Seagate 7200.10/.11 or a Western Digital Raptor. A raid 0 or 5 array would speed things up.

For your setup, I'd recommend Raid 5 or 0+1 ( also called RAID 10) since both can recover from loosing a drive (technically a RAID 10 set can loose two drives but only one set of striped drives). A simple RAID 0 (like I run in my gaming PC) will loose all of it's data if one drive fails.

That Caldigit card looks good. It's pretty much overkill for your uses but I took a look around and couldn't find a simple PCIe RAID card for the Mac.

One thing you may want to try doing before buying a $500-$1000 RAID card is create a RAID 0 array from the two 500GB drives and see if the performance satisfies your goals. If so, you may want to go for a RAID card afterall, if not, save yourself the $500...

 

[tyr2]

You could create a 640GB RAID 10 Volume using the disks you already have, by using a 320GB partition from each disk. You'd then have the remaining space on your first three disks to use up. You could create a 540GB RAID 0 scratch volume (using 180 GB from each of the 3 disks), this would be nice and fast but offer no data redundancy.

You might want to consider replacing the 320GB with a 500GB then you can create a 1TB RAID 10 volume, with 250GB space spare on the 750GB disk for storing junk on.

 

[mgsarch]

You could create a 640GB RAID 10 Volume using the disks you already have, by using a 320GB partition from each disk. You'd then have the remaining space on your first three disks to use up. You could create a 540GB RAID 0 scratch volume (using 180 GB from each of the 3 disks), this would be nice and fast but offer no data redundancy.

You might want to consider replacing the 320GB with a 500GB then you can create a 1TB RAID 10 volume, with 250GB space spare on the 750GB disk for storing junk on.

Thank you everyone.

Kitty, I am so jealous of your system. I'll read your post now.

Drobo is out for sure. The goal is to speed things up significantly and USB 2.0 would is a huge step back. I also will never trust the safety after multiple corruption/failures over the years.

Tyr, I think that's a pretty interesting solution. I think that's probably a great idea because I could pull the 320 and put it into the FW800 enclosure i have idling.

However, I think I might prefer RAID 5+0 to RAID 1+0? Would parity checking not be sufficient for a workstation that will also be backed up regularly? (on and off site). I certainly don't like not have any redundancy and increasing my risk by striping but with the parity checking I thought it might be perfect? 3x500 1x750. Maybe put the 750 in the DVD region and get another SE16/RE2.

What sort of "ineffeciency" would I see by pairing the Seagate 7200.11 with the WD SE16s? Seagate is faster and 32mb cache vs. 16mb.

Can you really partition the 750 to 500 and keep the 250 out of the raid array? I had thought the array nabs the whole disk.

 

[tyr2]

Thank you everyone.
However, I think I might prefer RAID 5+0 to RAID 1+0? Would parity checking not be sufficient for a workstation that will also be backed up regularly? (on and off site). I certainly don't like not have any redundancy and increasing my risk by striping but with the parity checking I thought it might be perfect? 3x500 1x750. Maybe put the 750 in the DVD region and get another SE16/RE2.

Raid 5+0 (50) would require 6 disks to achieve. You need three disks to do RAID 5, then twice that to stripe into RAID 50.

Theoretically you could do RAID 5 with 4 disks, still keeping one disk for parity. However the software RAID in OS X does not support RAID 5 at all. It only supports RAID 50 in the event that the RAID 5 devices are provided by some hardware RAID device.

If you're wanting to do RAID 5 you need the BTO RAID card.

What sort of "ineffeciency" would I see by pairing the Seagate 7200.11 with the WD SE16s? Seagate is faster and 32mb cache vs. 16mb.

Matching up the Seagate with the WDs I wouldn't expect a massive degradation in performance, or at least nothing you'd really notice. They're all SATA II / 3Gbps, i.e. they all support Native Command Queueing which IMHO is the most useful feature of SATA II. So I wouldn't worry about it.

Can you really partition the 750 to 500 and keep the 250 out of the raid array? I had thought the array nabs the whole disk.

Yes. I have 2 disks for OS X. Both are 500GB. I have a 300GB RAID 1 for boot and a 400GB RAID 0 for scratch, made from the 2 disks. There's no reason this wouldn't work for a RAID 10, you can just add partitions rather than entire disks. Unless you're using the BTO RAID card in which case it may well grab the whole card.

 

[mgsarch]

I can't believe I told you I wanted RAID 5+0!!!

I just wanted RAID5, no need for the mirror. I'm sad the software raid can't do it but that's ok, I've been preparing myself for the depression of spending money on this RAID card.

Not to be picky, any opinion / real experience on the added danger of using the desktop/consumer version of these drives as opposed to the server grade?

Like I had said, I'm pretty sure the only difference is that the firmware allows RAID error handling on the server drives. Hardware is the same. What happens on the consumer drives? They just keep trying?

I'm exhausted, sorry if I'm not being clear, I think I'll bite the bullet and order something on Monday.

Raid 5+0 (50) would require 6 disks to achieve. You need three disks to do RAID 5, then twice that to stripe into RAID 50.

Theoretically you could do RAID 5 with 4 disks, still keeping one disk for parity. However the software RAID in OS X does not support RAID 5 at all. It only supports RAID 50 in the event that the RAID 5 devices are provided by some hardware RAID device.

If you're wanting to do RAID 5 you need the BTO RAID card.



Matching up the Seagate with the WDs I wouldn't expect a massive degradation in performance, or at least nothing you'd really notice. They're all SATA II / 3Gbps, i.e. they all support Native Command Queueing which IMHO is the most useful feature of SATA II. So I wouldn't worry about it.



Yes. I have 2 disks for OS X. Both are 500GB. I have a 300GB RAID 1 for boot and a 400GB RAID 0 for scratch, made from the 2 disks. There's no reason this wouldn't work for a RAID 10, you can just add partitions rather than entire disks. Unless you're using the BTO RAID card in which case it may well grab the whole card.

 

[kiomon]

This is the raid card I was talking about.

http://www.caldigit.com/RAIDCard/

It connects internally like the Apple Raid card, so you just slide the 4 internal drives in, BUT it adds 3 expansion slots and thusly can grow over time. Its cheaper than the apple, will most likely perform better.

You definitely want RAID 5 or 6. I use RAID 5 and I have had drives die. I typically by OEM from Newegg, and test the drives for bad sectors then roll them in if they pass. Again may be overkill, but the card adds about $600, but gives you a TON of flexibility down the road, but you cant do software raid anyway for raid 5 (i wouldnt trust it)