Adding 1TB of SATA3 SSD with Apricorn and Highpoint RAID to my 2009 MP

Discussion in 'Mac Pro' started by VirtualRain, Dec 8, 2012.

  1. macrumors 603

    VirtualRain

    Joined:
    Aug 1, 2008
    Location:
    Vancouver, BC
    #1
    The problem:

    Like many of you frustrated by a lack of new technology in the Mac Pro, I've been looking around for the ideal SATA3 PCIe card for the Mac. My requirements were:

    • Bootable RAID0 (native OS X support)
    • SATA3
    • No bottlenecks (or PCIe x4 minimum)
    • Affordable (sub $200)

    Alas, such a card does not exist. There are lots of PCIe SATA3 cards with x2 bus interfaces (such as the Apricorn Velocity Solo x2), but thats going to be a bottleneck for a pair of current SSDs in RAID0. There are also very expensive RAID controllers from Areca and Atto but that's not what I want either.

    However, member All Taken recently put me onto the Highpoint 2720SGL which offers an 8x interface with native OS X support. The only problem with it was that it wasn't bootable in the Mac Pro (it is on a Hackintosh though). The reason this eluded me before, was the fact that Highpoint doesn't list it among their officially supported Mac products. The fact is, it works flawlessly without issue. There are no sleep or other issues I've come across after running it for a couple of weeks.

    The Solution:

    So while I ultimately wanted to have both my OS/Apps and Aperture Photo Library all on a single large SSD RAID0 volume, I recently decided this may never happen, so I opted to separate them.

    The solution I decided on was:

    • 256GB Crucial M4 on Apricorn Velocity Solo x2 for OS/Apps
    • 3x256GB OCZ Vertex4 on the Highpoint 2720SGL for Aperture Libraries

    My thinking was that my OS and Apps would benefit most from top notch small random read/write performance, which the M4 excels at. And, of course, the Aperture library (storing photos of 25-100MB in size) would benefit most from sequential read/write performance where RAID0 arrays excel. While I could have just purchased a pair of 256GB SSDs for the Aperture Library, I opted for a trio of drives to have more capacity and to really try to push performance. And, as you can imagine, that spare port on the 2720 is crying out for a fourth drive. :eek:

    I had purchased the Crucial M4 some time ago when the prices came down to around $1/GB and was running it as my boot drive off the SATA2 backplane. It was trivial to connect it to the Apricorn Solo x2 and drop it in the top slot.

    The rest all came together on Black Friday when the the 256GB Vertex 4 was available from Newegg for $175 CAD (it was $159 on Amazon at that time for US customers) and the 2720SGL was also on sale (and still is).

    Why the Vertex 4:

    I've been fairly critical of the Sandforce drives in the past, and, of course, OCZ has been one of the major vendors for Sandforce drives. On top of that, their reputation for using early adopters as beta testers is widely publicized. So why would I buy an OCZ product? Well, normally, I wouldn't for those exact reasons, however, in the case of the Vertex 4, a few things are different... (1) the drive uses a proven Marvell controller (2) the drive has been on the market for several months without any wide-spread complaints of issues (3) the performance is top notch (4) the price was rock bottom.

    I did a lot of reading on the Vertex 4 and there's no MLC based drive that can write faster. There's an interesting and cool reason for this. As you may know, SLC NAND is single-bit per cell technology and it's super fast to write data to although it's very expensive. MCL NAND on the other hand, is dual-bit per cell so it's much more affordable but it takes a lot longer to write data to, hence write speeds suffer. This (and the endurance benefits) is why you see SLC NAND typically used in enterprise drives. It seem OCZ took advantage of the fact that you can treat MLC NAND somewhat like SLC NAND if you're only writing one bit per cell at any given time, and thus gain some write performance. Of course, this will only work up to 50% of the drives capacity. After which, the garbage collection needs to reorganize the data from single bit per cell to double bit per cell to free up NAND. This is pretty effective if you never write more than 50% of your free capacity at any given time. You effectively get SLC write speeds using MLC NAND when writes are not more than 50% of available free space.

    While this was not a factor in my decision, I think it's a novel approach to improving performance without resorting to compression.

    Additional hardware:

    Besides, the Highpoint 2720 card, I also needed a couple of cables, and some way to mount the 3x Vertex 4 SSDs.

    Here's the other stuff I purchased:


    Installation:

    The M4 on the Apricorn Solo X2...

    [​IMG]

    The Vertex SSDs in their mounting bracket... note, there's room for 1 more :)

    [​IMG]

    [​IMG]

    In the Mac Pro optical bay carrier...

    [​IMG]

    The Mac Pro interior with cables running from the 2720 to the optical bay... (the idea for routing the cables comes from ConcordRules thread - Thanks!)

    [​IMG]

    Everything installed...

    [​IMG]

    Setting up RAID on the Highpoint 2720:

    OS X recognizes the 2720 as a Parallel SCSI/SAS controller...

    [​IMG]

    Drives will show up as SCSI/eSATA drives with an orange icon. You can manage them in Disk Utility, just like any other drive. And therefore you have all the software RAID options at your disposal that OS X natively offers.

    [​IMG]

    If you want to take advantage of other RAID modes the 2720 offers, you can do so... you simply need to install their web RAID management interface (for the 2722) to allow you to access the card's firmware through a browser. I tried this and it works fine. However, it's RAID0 stripe size options were limited so one option (32K IIRC) so after playing around with it, I rebuilt my final RAID array using disk utility. More on performance and stripe size options coming up.

    PCIe Slot:

    All the recent Mac Pro's have a pair of 4x (electrical) slots at the top that connect through the ICH and a pair of 16x slots at the bottom connected to the X58 IOH.

    I believe the top two x4 slots are switched... in other words, even though cards in both those slots will negotiate a x4 connection, they will be sharing that x4 (2GB/s) bandwidth. At any rate, what I found was even more damning... it appears there's something in the architecture of the Mac Pro that's limiting bandwidth of those top slots to 1GB/s or about half what I expected.

    The 2720 with 3x SSDs in one of the top slots could not sustain transfers more than about 1000MB/s. In one of the x16 slots, I could get sustained transfer rates around 1370MB/s which is more in line with the limit of the three drives in RAID0.

    Since I have a pair of recent Apple displays with mini display port connectors that I'm driving with a pair of GT120's, I'm currently running the Highpoint in one of the crippled x4 slots but I hope to rectify that soon once I get a single slot graphics card with dual MDP ports.

    Drive benchmarking tools:

    These are the drive benchmarking tools that I'm familiar with:

    • Xbench – (free) last updated in 2006 before SSDs and notoriously inconsistent
    • AJA – (free) sequential read/write tests only and doesn’t appear to work on boot volumes
    • Blackmagic – (free in the app store) sequential read/write tests only
    • Disktester – ($40) No comment
    • Quickbench – ($15) offers sequential and random tests for a reasonable price

    I purchased Quickbench. It seemed like a small investment compared to the hardware and allowed me to test a number of things to greater depth than any of the free tools. Having said that, the sequential speeds reported by AJA, Blackmagic, and Quickbench are all consistent which is a good sign and a solid endorsement for the free tools if that’s all you need to test.

    Optimal stripe size:

    I've always subscribed to the school of thought that stripe size should be a function of the type of data you're storing/accessing... a small stripe size for smaller files... and larger stripe sizes for larger files. When SSDs came along, the write amplification penalty due to large NAND page and block sizes also supported the use of larger stripe sizes. So when I started into this project, I was fully expecting to run as large a stripe size as I could.

    In my research however, I found another guy that had been testing various RAID0 stripe sizes with the Vertex 4 (on a PC) and found that they performed better with smaller strip sizes... right down to 4KB. I decided to do a bunch of stripe size testing of my own before committing to one on my array.

    Using the x16 slot for my testing, I tried 128K, 64K, 32K and 16K (the smallest OS X RAID0 allows) and the results were consistently better with decreasing stripe size. I forgot to take a screen shot of the benchmark tool on the larger stripes, but if I recall correctly, the differences were significant between 128K and 16K for larger sequential transfers (on the order of around 100MB/s but still less than 10%) and insignificant at lower 4k transfer sizes.

    OS X vs Highpoint RAID0:

    I compared RAID0 performance using the 2720 firmware vs. Disk Utility in OS X and results were identical in both scenarios.

    Single vs Dual vs Triple SSDs in RAID0:

    I also opted to test the Vertex 4 and 2720's ability to scale as drives were added to the array. Here are the max sequential read / write speeds in each case:
    Single Drive: 489MB/s / 465MB/s
    Dual Drive RAID0: 956MB/s / 927MB/s
    Triple Drive RAID0: 1066 MB/s 1370 MB/s

    Note that the read speeds don't scale well linearly beyond two drives. I'm not sure if this is the card or the drives. However write speeds do scale well as drive count increases (at least up to 3 drives).

    Final Configuration:

    For simplicity, I opted to setup the final 3x 256GB RAID0 array in disk utility with a 16K stripe.

    Here’s the benchmarks: (attached)
     

    Attached Files:

  2. All Taken, Dec 8, 2012
    Last edited: Dec 8, 2012

    macrumors 6502a

    Joined:
    Dec 28, 2009
    Location:
    UK
    #2
    Nice post dude, you see that area next to the CPU heatsink? The large area that looks more than comfortable for an additional 4x SSD's? ;)

    That area or the use of a DX4 will make full use of the card. If you decide to put some by the heat sink on the empty daughterboard space be sure to place a piece of plastic under the SSD's.

    When you route the cables you could unscrew the two screws securing the PCI fan and route the cables behind and under it.
     
  3. hfg
    macrumors 68030

    hfg

    Joined:
    Dec 1, 2006
    Location:
    Cedar Rapids, IA. USA
    #3
    Very nice writeup ... thanks for sharing! :)

    To make the AJA work on boot drives, use "Get Info" to add yourself to the list at the bottom, with read/write permissions, and click the "gear" and make yourself the owner.

    Keep us posted on your progress with this project.




    -howard
     
  4. macrumors 6502

    Joined:
    Jul 19, 2007
    #4
    Very great information and a really nice rig you got there (thanks for the photos as well). I did something very similar except I used 4 SSD's in the optical bay. I created a RAID 0 on two SSD's each. I used the same Highpoint cable (routed the same way you have shown) and power splitters. I didn't have a need for an optical DVD drive. The inspiration came from Concord Rules as well.

    mokeiko
     
  5. thread starter macrumors 603

    VirtualRain

    Joined:
    Aug 1, 2008
    Location:
    Vancouver, BC
    #5
    Thanks guys. It was a fun project and should keep the upgrade itch away now until a new Mac Pro reveals itself. :)
     
  6. macrumors 65816

    odinsride

    Joined:
    Apr 11, 2007
    #6
    Great writeup, I love reading things like this :cool:
     
  7. thread starter macrumors 603

    VirtualRain

    Joined:
    Aug 1, 2008
    Location:
    Vancouver, BC
    #7
    BTW, I'm interested in knowing if anyone else has hit up against a 1GB/s bottleneck using one of the top x4 PCIe slots in a 2009+ Mac Pro.
     
  8. macrumors newbie

    Joined:
    Jan 19, 2013
    #8
    Bootable?

    Hello VirtualRain
    ITs the raid configuration bootable?
    i want put only the 2720SGL with 2 sad
    thanks a lot for you attention. Sorry for my english
     
  9. thread starter macrumors 603

    VirtualRain

    Joined:
    Aug 1, 2008
    Location:
    Vancouver, BC
    #9
    No, the Highpoint is not bootable.
     
  10. macrumors 68030

    wonderspark

    Joined:
    Feb 4, 2010
    Location:
    Oregon
    #10
    The four SSDs in the ODD bay... do you have two of the SATA power splitters, sending two to a pair of SSDs, and the other two to one SSD and the optical drive?

    ----------

    And, do they give you any trouble, power-wise?
     
  11. macrumors newbie

    Joined:
    Jan 19, 2013
    #11
    thanks a lot
     
  12. thread starter macrumors 603

    VirtualRain

    Joined:
    Aug 1, 2008
    Location:
    Vancouver, BC
    #12
    I have three SSDs in the optical bay and just one of the aforementioned 1 to 3 SATA power splitters. And no, they don't give me any trouble.
     
  13. macrumors 68030

    wonderspark

    Joined:
    Feb 4, 2010
    Location:
    Oregon
    #13
    Ah, yes. I forgot that thing splits into three.

    I'm figuring that the Superdrive pulled about 30 Watts from one of the plugs, as it says 5v/12v === 2.0A/2.5A on the unit itself. Assuming the 12v/2.5A is the power draw, that equals 30W. The Samsung 830 SSD draws 0.24W, and the new 840 Pro SSD draws only 0.068W during active use, so even using the less efficient 830, 30 Watts would cover 120 SSDs, so I'm sure four of them on a splitter is totally insignificant.

    This leads me to imagine replacing my 8-bay 3.5" HDD RAID box with 12 SSDs inside the ODD bay instead, all connected to my 12 internal ports on my Areca RAID card in RAID 6. I think it would probably saturate the x8 lane RAID card, haha! Too bad we're not in the days of cheap 1TB SSDs. :p

    ----------

    Looking around, I'm seeing all kinds of variable reports on power consumption, but none are more than a half Watt.
     
  14. macrumors member

    Joined:
    Mar 29, 2010
    #14
    anybody can point me to get long sata cable to reach ! sata3 controller ? :)
     
  15. thread starter macrumors 603

    VirtualRain

    Joined:
    Aug 1, 2008
    Location:
    Vancouver, BC
    #15
    Look on Newegg for SATA cables 1m in length. There appears to be plenty to choose from.
     
  16. macrumors 6502a

    Tucom

    Joined:
    Jul 29, 2006
    #16
    Is that ECC RAM? There's no heatsinks on the RAM :confused:
     
  17. thread starter macrumors 603

    VirtualRain

    Joined:
    Aug 1, 2008
    Location:
    Vancouver, BC
    #17
    Yes it's ECC sticks from OWC. I'm surprised you're surprised by the lack of heat spreaders. Stock RAM doesn't include heat spreaders either. Heat spreaders on RAM are more about cosmetic bling and marketing than cooling.
     
  18. macrumors 68000

    Joined:
    Sep 12, 2007
    #18
    Just for the record, AJA System Test will let you bench a boot drive (See my thread in fact!).

    You just need to change the boot disk permissions to allow anything to write to it.
     
  19. macrumors member

    Joined:
    Mar 29, 2010
    #19
    thanks friend ,
    but what about the black combo cable there connected to SATA3-card ? is it the Apple(bundled) or you buy it ?
    gr8 system you got there , cheers ! nice work
     
  20. macrumors 6502

    Joined:
    Jan 17, 2012
    #20
    The combo cable is terminated in two separate (Power and SATA) connectors at the mother board, I used the power portion of the combo cable only, splitting it at the optical bay using a 1 to 3 splitter adaptor. I disconnected the SATA connection of the combo cable from the mother board and ran a standard SATA cable from the mother board to the bay. Additional SATA comes from the PCIe card.
     
  21. macrumors member

    Joined:
    Mar 29, 2010
    #21
    got it thanks ! you rock ! :)
     
  22. macrumors newbie

    Joined:
    Dec 1, 2011
    #22
    Hello VirtualRain!
    Tell me please, with Apricorn Velocity Solo x2 in system info-hardware-serial-ata, what series chipset shows: unknown or other? Is bootable in Mac OS and Windows? And what version AHCI supported?
    Thanks!
     
  23. thread starter macrumors 603

    VirtualRain

    Joined:
    Aug 1, 2008
    Location:
    Vancouver, BC
    #23
    The Solo x2 boots natively in OS X. I can't comment on how it behaves in Windows... I despise Windows and will never let that sad excuse for an OS ever reside on any of my systems... ever again! :D I'd be completely remiss if I didn't aggressively discourage you from using Windows of any kind on your Mac!

    Here's how it appears in Sys Info...
     

    Attached Files:

  24. macrumors newbie

    Joined:
    Jan 1, 2015
    #24
    The Photos Not Showing Up

    Virtual Rain - I am not able to view the photos in your post (They are showing up as question marks.). Please advise. Thanks!
     
  25. thread starter macrumors 603

    VirtualRain

    Joined:
    Aug 1, 2008
    Location:
    Vancouver, BC
    #25

    Yeah sorry but in the last couple of years I've switched photo hosting providers. If I have those with my current provider I'll update the post.
     

Share This Page