Mac Pro - Bootable RAID Cards

Discussion in 'Mac Pro' started by 50thVert, Jan 18, 2009.

  1. 50thVert macrumors regular

    Jun 20, 2005
    St. Louis, MO
    Hello all,

    The time has come to move to a RAID setup in my Mac Pro. Ideally I'd like to run 4 1.5TB SATA drives in a RAID 5 or 10. It seems there are a LOT of threads on this topic all filled with different information, so I have a few questions...

    Which RAID cards are bootable?

    Do any work like the Apple card and interface with the drives through the logic board?

    If not, how do you connect the internal drives to the RAID card in their stock bay locations?
  2. nanofrog macrumors G4

    May 6, 2008
    Do you want to be able to run multiple OS's, or stay strictly with OS X?

    It could help direct you to the right card. Other information would too.
    • # & type of drives (Expansion: will you need to increase #, or use SAS)
    • RAID Type (change in near future?)

    Personally, I really think you'd be better off using a separate OS drive, and mount it in the empty optical bay. You need a Mini SAS Fan Out cable to attach it to the logic board. Saves you a massive amount of headache if something ever goes wrong, and it will boot faster, as you don't have to wait for the RAID card to initialize.

    A short list of manufacturers to consider:
    Accusys (ODM for CalDigit, available at newegg last I saw)
    None that I'm aware of.
    The Mac Pro drive sleds SATA connectors all attach to a single connector that plugs into the logic board. It's called a Mini SAS or SFF-8087 connector.

    You unplug this cable from the board, and plug it into the RAID card.

    Please note, that in a Mac, the existing Mini SAS cable length can be an issue (too short), particularly to mount it into Slot 1 or 2. particularly important if the card is an 8x lane PCIe model. It can be gotten around, but requires some imagination, and DIY skills. Particularly with a soldering iron. I've known someone to solve it with gaffer tape, but that one seems really untrustworthy over time, IMO.
  3. 50thVert thread starter macrumors regular

    Jun 20, 2005
    St. Louis, MO
    I run OS X and XP inside of VMWare. Being bootable into multiple OS's is not of concern to me. I will run only SATA II drives because of cost. Ideally I'll be starting with 3 1.5TB drives in a RAID 5, but would like to be able to expand past that. Possibly up to 6 drives total.

    Maybe I'll keep my Velociraptor after all. ;) It's a bare 2.5" drive though, can you still mount those up in the optical bay? How well does Time Machine work on a RAID setup? I.e. if my VR fails can I restore from a TM backup on my RAID?

    I'm familiar with a few of those names. Which ones have backup batteries? I have my whole setup connected to a UPS but I don't want to risk my data in that respect.

    Yet another thing I had no idea about. :) I'd definitely spring for a longer cable, buy once, cry once, and do it right!

    So from that expansion on my original post, which card would you recommend?
  4. nanofrog macrumors G4

    May 6, 2008

    Any of the cards listed above will handle your needs from what I understand ATM. I'm glad you took the warning about booting off the array seriously. :D

    Makes card selection easier. Possibly cheaper too. :D

    I wouldn't recommend using consumer grade drives. They just can't handle the abuse, and fail prematurely. Use enterprise drives. When using a RAID card, look for an HDD Compatibility List, as the card manufacturer has already tested drives. It can prevent discovering what you wanted and bought won't work. :(

    If you want to go to 6 drives or so, you'd either have to move the optical drive externally, or place the array externally. (The optical drive move is cheaper I think). :p External RAID enclosures are not. :(

    Also, do not try to span an array between both internal and external drives. That's a can of worms that should be left unopened. Forever. :eek: :D
    With a little creativity, absolutely. If you have an old 5.25" CD-ROM (dead is ideal), you take it apart, and use the metal case. You'd need to drill a couple of holes in the metal shell to mount the 2.5" drive. (Another member has posted success with this method). :) I like etched PCB material, as it's an insulator (extra insurance ;)), but there's no need to have to go buy some of the specialty 3rd party offerings. Way too expensive for what it is.

    Just take a good look at the photos, and you'll see what I mean. :p

    Time machine would work, but I'd use an external drive for it, or better yet, either a SAN or NAS, as it can have the capacity needed to backup the array as well. Data is expensive to recover. An OS install can be cloned, or in worst case, reinstalled from the original media. Just takes longer. :p

    Good to hear about the UPS. What size is it, BTW?
    You don't want one that's too small, as it's effectively worthless. :(

    As far as batteries:
    HighPoint: A couple of models do.
    Areca: Yes, but not recommended by their support personel. :eek: The UPS offers much better protection, as a power outage can wreak havoc.
    Atto: They list one, but don't actually sell it.
    CalDigit: Yes, they offer one.
    Accusys: IIRC, they make one, but I don't know if you can find it. Small presence yet in the US. Not sure about overseas.

    Here's where it gets a little complicated. The data end can actually reach, if you mod the power connections. They're what keep it from reaching. Unfortunately, Apple doesn't offer a longer version, and this is why I made the soldering iron comment. :D (I don't trust crimp connectors in this sort of situation, and resort to solder and heat shrink tubing). ;)
    I'd take a look at the HighPoint RocketRAID, CalDigit, or Accusys for the budget end.

    Areca and Atto offer other features that are worth it for some. Not only amazing speed, but some additional redundancy features the others don't, like a backup copy of the Partition Tables. Extremely handy in case of a failure.

    Please make sure you do your research before you buy, or have to deal with major headaches and returns.

    Now time to digest this. RAID = information overload. :p
  5. 50thVert thread starter macrumors regular

    Jun 20, 2005
    St. Louis, MO
    Noted. :)

    Maxupgrades has a setup that mounts two 3.5" drives into the spare optical bay, leaving the filled one alone. I'll probably go that route and mount my VR and the stock 320GB drive up there for a TM backup of my OS drive.

    Also noted, haha.

    I'm running a APC Back-UPS RS 1500VA. It should be able to handle my setup. :p

    Interesting that Areca doesn't recommend it. Maybe I'll forgo the battery after all and just rely on the UPS.

    Eh, I'd rather not mess with it and just buy a longer cable. Can you find them on Newegg?

    I think I'll check out some HighPoints and some Arecas. How big of a speed difference is there between the two with RAID5?
  6. nanofrog macrumors G4

    May 6, 2008
    From what I could tell in the photo's, you could fit 2x 3.5" drives and the 2.5" VR on the bottom plate from an optical drive.

    The offering from MaxUpgrade will work, and is essentially the same thing. Just expensive for a metal plate and a power splitter.

    Up to you of course. :D

    Perfect. :D

    That will work. :)

    I just skipped it myself, as have others I'm aware of. ;)

    To swap the cable out, you'd need to get a Mini SAS to Backplane Fan out cable. The connector has both data and power, and is held by 2 screws in the back of the sled area. (2 metal tabs).

    The problem is, Apple's cable is proprietary. :mad: Specifically the power connector that attaches to the logic board. To do it this way, you'd still have to cut the power connections loose, and attach to the new cable.

    The additional drives (optical bay), would be much easier. You can use off the shelf cables. :)

    Either way, a mod is necessary, not optional. :(
    Areca designs and builds at least some of HighPoint's cards, and those that are similar (based on an Areca design), are missing features and not the same performance.

    If you want those features, and the fastest possible data throughput, the Areca is the way to go. Not exactly cheap though. In your case, don't go past 8 ports, as it drives up the cost. Say ~$125 - 150 per Mini SAS connector.

    Given the physical installation (space & cables), I opted to build. Currently under way. ;) Just waiting for Intel to release the Xeon 5500 series/3500 series parts. And mother boards of course. :p

    I'll just have to deal with hacking it to make OS X work. :eek: :D
  7. 50thVert thread starter macrumors regular

    Jun 20, 2005
    St. Louis, MO
  8. nanofrog macrumors G4

    May 6, 2008
    The Max Connect won't really help. It's just a female to female Mini SAS adapter. It lets you plug the Mac's connector (Mini SAS) to a Mini SAS fan out cable. The 4x SATA connectors then are attached to a RAID card that doesn't use Mini SAS connectors. The HighPoint RR2640 series (Fake RAID) has one like this. (Just single drive SATA connectors).

    The connector on the Mac isn't actually proprietary. It's used in RAID systems though, and but an uncommon method of connecting drives directly to the main board.

    I avoid the ODD_SATA Ports (5 & 6), as they can be a pain. However, you may be able use one or both for OS drives. I'd still get a Mini SAS fan out cable, attach to the logic board, and run them that way. They'll will boot. ;)

    If you want to add an additional drives/s, (5 through 8) to the array, you would also need a Mini SAS fan out cable. Assuming the card can run 8 drives internally.

    Please note, that the CalDigit can only operate 4 drives internally. Additional drives were meant to be attached externally, and you must use CalDigit's HDElement drive enclosure. Not the least costly option, as you're locked into their products. If you do go this way, you can at least upgrade the drives. More $$$ of course. :p

    They come equiped with consumer HDD's (Hitachi, which is beyond lousy for customer support. There isn't any. You're on your own). :mad:
  9. 50thVert thread starter macrumors regular

    Jun 20, 2005
    St. Louis, MO
    I'm confused. At this point everything is figured out except the cabling. To run the 2 optical bay drives to the logic board I need a standard mini-SAS fan out cable. And to run the 4 backplanes drives to the raid card with minimal resistance I will buy a maxconnect and a mini sas cable. Is this correct?
  10. nanofrog macrumors G4

    May 6, 2008
    Sort of. :eek:

    You do need a fan out cable to go from the drives in the optical bay to the logic board. :)

    As for the rest of it, it depends on the actual RAID card used. If you went with a card that uses standard, single connection SATA ports, then Yes, you would need the Maxconnect adapter and another fan out cable.

    You're better off getting a RAID card that uses the Mini SAS port. Common amongst true RAID cards (I/O processor & cache). You then attach the drive cable (data end) originally located on the logic board, and plug it into the RAID card.

    This is the only way to make this way work. Usually quite easy. But in the Mac Pro's case, it gets a little more complicated. Thier power connections are too short to allow the cable to reach slot 1 or 2 easily, and tends to be short. (Depends on the card length).

    Hence the need to hack the power portion of the backplane cable. (Another type of fan out, but the power connector makes it proprietary, as they don't get power off of a Molex or SATA power connector. They use a 6 pin cable instead).

    It's not that it's extremely difficult, and there's no need to be terrified. Get a length of black wire, cut the power cable (single conductor at a time), slide on some heat shrink ( out of the way), solder, and finally heat the shrink tubing. It really isn't daunting, particularly if you do it one wire at a time. No way to get anything crossed. ;)

    It won't take too long, but it does require some time and a little effort, rather than just plug in an off the shelf cable. You could use crimp connectors, but be careful that they aren't in a bend, or can short.

    Hope this helps. :)
  11. 50thVert thread starter macrumors regular

    Jun 20, 2005
    St. Louis, MO
    Ugh. Apple has to make things so difficult. :rolleyes:

    I'm looking at the ARC-1210, which has a true IO proc and cache and uses 4 individual SATA ports.

    Saying I look at a card with internal mini-SAS ports, why wouldn't the MaxConnect still work in the case? They make a model with a mini-SAS cable: :confused:
  12. nanofrog macrumors G4

    May 6, 2008
    Not sure why sometimes, but it seems to happen often. enough to be a pain. :p
    It will work, but the Maxconnect adapter and fan out cable combination would be required.

    If you wanted to use the Mini SAS connector, take a look at the ARC-1231ML. It supports more drives than you need, but you can get an idea of what fast is. :p
    Do you mean to use the Maxconnect to attach another cable (Mini SAS to Mini SAS) as an extension to reach?

    If that's the case, then yes, it should work. :)

    In the case of the ARC-1220, you would use the Maxconnect adapter with a fan out. It doesn't use a Mini SAS connector.

    Usually though, adapters aren't used internally, as they needed. Externally, they can be useful, but can cause problems (internal to external adapters). They can cause the array to become unstable. :(
  13. 50thVert thread starter macrumors regular

    Jun 20, 2005
    St. Louis, MO
    *sigh* Finally, I have my answers. Thanks very much for all your help man! :D

    Oh and a whopping $500 difference between the ARC-1210 and the ARC-1231ML! :eek: How much of a difference in speed is there realistically going to be between the cards running 4 1.5TB drives in RAID 5?

    Also, since the Mac Pro can't hold more than 6 hard drives and one optical drive internally, I don't see my array growing past 4 total drives.
  14. nanofrog macrumors G4

    May 6, 2008

    I wasn't quite sure what you were asking, so I was giving general info. It was when you gave me a model #, that made it easy. :D
    Not enough to worry about. :) Between these two, the cache makes a difference, but I don't think it's worth the $$$ difference given your set up. With more drives, it would be another story. ;)

    Easier on the wallet is a good thing. :D
    For 4 drives, the ARC-1210 will serve you well. :)

    Though the SATA based RAID cards are more forgiving with what drives they'll work with, do take a look at Areca's HDD Compatibility List (direct download link to a .pdf file) before you buy any drives. (Incredibly important if you go with a SAS RAID card like the ARC-1220 or ARC-1680 series).

    Hope this helps, and good luck. :)
  15. frimple macrumors 6502

    Nov 18, 2008
    Denver, CO
    Hey guys, thanks for this thread. I've got a question for you nanofrog, what about RAID cards that you can boot off of?
  16. 50thVert thread starter macrumors regular

    Jun 20, 2005
    St. Louis, MO
    Well upon further investigation, I think I may be forgoing the ARC-1210 and going with a RocketRAID 3510. It's a $40 difference on Newegg, but it has the same faster IOP341 processor (800MHz) and the DDR2-533 cache of the ARC-1231ML, only it's fixed at 256MB. It also has only one mini-SAS port which satisfies my need for 4 SATA channels and minimizes the cabling. Seems like it's worth the $40, yes? :)

    And seeing as the 35xx series is OS X bootable, I'm considering selling my Velociraptor & MaxUpgrades MacPro VR drive sled, upgrading to the 3520 (2x mini-SAS for 8 SATA channels), and running 6 1.5TB drives in a RAID 6 with a hotspare.
  17. 50thVert thread starter macrumors regular

    Jun 20, 2005
    St. Louis, MO
    It looks like HighPoint has plenty of RAID cards that are OS X bootable. Check out their Mac-oriented website. :D
  18. frimple macrumors 6502

    Nov 18, 2008
    Denver, CO
    Is there a definitive list? I was looking for something more along the lines of an Areca card
  19. nanofrog macrumors G4

    May 6, 2008
    Are you looking for OS X only, or do you need to run a multiple OS environment?

    Other questions:
    How many drives/type (SATA, SAS, or both)?
    What type of RAID do you need?
    Expansion considerations?
    Internal, external, or both? (Please don't try to mix internal and external drives into a single array).

    HighPoint : Boots, but only a single OS

    These have models that can boot multiple OS's:

    Sorry if this is information overload, but it comes with the territory when you get into hardware RAID. ;)
    I'm aware of HighPoint's site (dual). One for general (PC, Linux, OS X), and one aimed specifically at Macs. :D

    If you go with a separate OS drive, and I seriously can't stress this enough, you won't actually need to boot from the RAID card.

    You might want to take a look at newegg's customer review section on the 3510 first, before you order it.

    If you're OK with it, saving $$$ is nice. Just don't shoot yourself in the foot as it were, by purchasing something that won't suit your needs.
  20. 50thVert thread starter macrumors regular

    Jun 20, 2005
    St. Louis, MO
    Read through the reviews for the 3510 and the majority of issues seem to be stemming from motherboard incompatibility issues from home-built systems and Windows Vista. :rolleyes: The 3520 (same card with another mini-SAS port) seems to get good to great reviews and I have a feeling the lower price point of the 3510 is inducing some poor reviews due to the user-error. :p

    I really do understand the appeal for running a single OS drive, especially in the case of a RAID card failing. But I'm being swayed towards the 3520 and booting from the RAID card simply because I have the option to expand the array to 6 internal drives, making both my OS and data access faster. Five 1.5TB drives in a RAID 5 would FLY and having an extra drive as a hotspare gives me enough peace of mind.

    Nevermind the fact that the bare VR and MaxUpgrades sled setup is worth a pretty penny. Enough to cover the two 1.5TB drives that I need to run a barebones RAID 5 (already own one).

    Or maybe I should just keep the VR as my system drive, get the 3520 and run a 4x1.5TB RAID 5 with a hotspare...

    How much different is the throughput on a 4 vs 5 drive RAID 5?

    The jury is still out. Got some thinking to do! :)
  21. frimple macrumors 6502

    Nov 18, 2008
    Denver, CO
    I will need to boot multiple OS's, however a majority of the time I will run the others through fusion in OS X. So the three are OS X (of course), Linux and Windows (Server 2008 and Vista).

    The drive setup is as follows:

    4 X25M's in RAID 0 (these will be the boot/app drives.. they format down to 75 each so I should get a total of 300gigs to work with across all the OS's/VM's)

    4 Raptors in RAID 5 (maybe 6) for data/workspace

    All internal, 2 X25's will fit into the space of one 3.5 so the plan is to use those 4 in the second optical bay. I've already resigned myself to making a custom housing for them. The 4 raptors will go in the 3.5 sleds in the main housing.

    So I'm looking for a card that can support the boot array as well as the storage array. Thoughts and suggestions are welcome.

    Thanks for helping me hammer this stuff out! I'm slowly accumulating the parts (got the x25's... raptors next) for this dream machine and I'd love to hear any thoughts you have! :)
  22. nanofrog macrumors G4

    May 6, 2008
    I read them, and wanted you to. Not to dissuade you, but to point out a few things. ;)

    Primarily what stood out to me, were the complaints of drives dropping out. This is the type of behavior that happens with incompatible drives. Hence the need to check the card manufacturer's site for their HDD Compatibility List.

    RAID cards don't automatically work with any drive. I would also expect those who had issues, were using consumer drives, not enterprise drives. Combine this fact with they can go from just fine to DOA in under 2 months, is a really good reason to avoid them. This is why the enterprise drives are worth the additional $$. Fewer problems. And, ultimately, less expensive when you consider the need for returns, down time, and the costs associated with diagnostics, etc. Especially for a company. I'm assuming you will be the one who does all the support/repair, so there's no cost in this regard. But the $$ for returns adds up fast.

    The 3520 will be fine, and it has the ability to run enough drives, since you would have a hard time fitting in any more than that. It's a Mac Pro, not a warehouse. :p
    It won't boot faster. :eek: The card has to initialize first, then load the OS. If you look around, you might notice boot times of a single drive vs. an array. The single drive is faster, and typically can boot in ~40 - 45 sec. The array can add 30s + to that time. I've seen an array take over 2 min to boot. :eek:
    If you go with the Velociraptor, go with the WD3000GFLS (cable connect version), and use the mount from Maxupgrades, if you want to install it in one of the sled locations.

    It's cheaper (~$250 with the mount), vs. the WD3000HFLS (backplane model) that goes for ~$310.

    If you're creative, you can mount it elsewhere with either a DIY means (decent, not tape or anything ;)). Noise Blocker X-Swing might help as well. You mount it under one of the 3.5" drives attached to a sled. You might need to keep slot 4 unused though. You'd have to check for clearance on this.

    PCB and plexiglass are good materials to use for DIY mounts, and can be found online for sure, and maybe locally, depending on where you live. Even Radio Shack carries breadboard (sort of a tan color PCB with a lot of holes).

    Remember, the Velociraptor is a 2.5" drive, and it Does Not need to remain in the aluminum carrier/mount. WD even sells it without any mount at all, as they don't run so hot it's actually needed. (Small is easier to find space for anyway). :D
    I would use the Velociraptor. (That's what I have, BTW). ;) You won't be disappointed. :D

    If you don't leave the system unattended for days, you won't need the hot spare. Though nice, the space available in the MP is at a premium as is, and you've already indicated you need every bit you can get for more than 4 drives.

    Moving the optical drive into an external enclosure is also a viable possibility.

    If you have an old 5.25" drive (CD, DVD, etc), you can use the bottom plate from that to mount the drives on rather than use Maxupgrade's optical bay mounting system. It's very similar, and quite expensive for what it is.

    DIY skills will pay off if you're comfortable doing it, and it isn't hard.

    2.5" drives can also free up some additional space. You can squeeze 4 into a single 5.25" bay. (Hot swap backplanes for 2.5" drives exist, and fit in a single bay).
    4 drives should produce ~300MB/s realistically.
    Add 75MB/s for each drive additional.

    A rough formula would be as follows:

    (Sustained drive throughput SDT * number of drives n)*0.85 = ~RAID 5 throughput

    I based the #'s on:
    SDT = 88MB/s (typical of current enterprise drives)
    n = 4

    Keep in mind, this isn't exact, but gives a realistic estimate. In some cases, the % can be as low as 75% (0.75).
    Research, research, and more research. :p
    Highpoint won't work.

    You'd need to use an Atto or Areca. Atto is more expensive, so I lean towards the Areca. Not to mention, it's faster. :eek:

    Take a look at the ARC-1231ML. 12 ports, not 8 (bare min for your needs), fast IOP, and has the ability to upgrade the cache via a DIMM. Not exactly cheap at ~$650 or so, but would scream with the drives you're considering. It also has the ability to operate at RAID 5 and 6. Not to mention some other useful features, like a Partition Table backup. (You will truly love this feature if something ever goes wrong, as it will save your butt).

    I would truly recommend going with the separate OS drive. (See previous posts). ;) A Velociraptor would work well for this.

    In reality, the SSD's aren't needed, and have a limited number of write cycles. ~100k writes with current Flash technology. I'm not a big fan of RAID 0, and given the write cycle limitations, I absolutely would not recommend this. In this case, figure 25k writes, then FAILURE. Just too much money to throw away. You'd be better running the drives independently as OS drives (1 per) than the RAID 0 route.

    Velociraptors are good drives, but you might want to look at WD's RE3's if you need greater capacity(Raid Edition 3). Fast, enterprise drives, and come in large sizes, up to 1TB. WD is also still providing some level of customer support. Other manufacturers have completely eliminated it, leaving the end user swinging in the wind. Hitachi... Oh, did I say that one out loud? :eek: :p Seagate's almost there too, even though their ES.2 enterprise drives are decent. :(

    I'd recommend RAID 5 exclusively for its balance of speed and redundancy, even for an OS array (if you do go this way). At least, it will still work in a degraded mode (slow) if a drive goes out, and you won't lose the data.

    If you wonder why, check out the costs for data recovery. The last quote I got on an 8 drive array was well over $20k! :eek: And this quote is recent (~2.5 months ago). :(
  23. ericjohnson1981 macrumors newbie

    Dec 15, 2008
    That's a great list [​IMG] [​IMG]
  24. frimple macrumors 6502

    Nov 18, 2008
    Denver, CO
    Thanks for the feedback and info! So on the Areca website I saw this note on the drivers page "Mac OS X (no_bootable)" (I have seen this on some other ones too). So this card is bootable for OS X?

    I might go the 4 separate OS drive route. And I've got the SSD's now so I've gotta find a justifiable purpose for them (before my GF sells them on Ebay). Regarding the life span of these drives here's a quote from the Anandtech review.

    Sounds sufficient for me, plus with the 3 year warranty on the drives that fits my timeline for replacement perfectly. Plus the possibility of > 600Mbs reads / 200Mbs writes just sounds sexy.

    The goal of this entire system is raw speed with large files (if you couldn't tell) and multiple VM's running at once. I'm not too worried about data loss as everything will be backed up daily to the NAS.

    EDIT: I wanted to add a link to the Anandtech review for those who haven't read it.
  25. hayduke macrumors 65816


    Mar 8, 2005
    is a state of mind.
    What software will you use for the back-up to the NAS?

Share This Page