Register FAQ / Rules Forum Spy Search Today's Posts Mark Forums Read
Go Back   MacRumors Forums > Apple Hardware > Desktops > Mac Pro

Reply
 
Thread Tools Search this Thread Display Modes
Old Dec 8, 2012, 07:06 PM   #1
VirtualRain
macrumors 603
 
Join Date: Aug 2008
Location: Vancouver, BC
Adding 1TB of SATA3 SSD with Apricorn and Highpoint RAID to my 2009 MP

The problem:

Like many of you frustrated by a lack of new technology in the Mac Pro, I've been looking around for the ideal SATA3 PCIe card for the Mac. My requirements were:
  • Bootable RAID0 (native OS X support)
  • SATA3
  • No bottlenecks (or PCIe x4 minimum)
  • Affordable (sub $200)

Alas, such a card does not exist. There are lots of PCIe SATA3 cards with x2 bus interfaces (such as the Apricorn Velocity Solo x2), but thats going to be a bottleneck for a pair of current SSDs in RAID0. There are also very expensive RAID controllers from Areca and Atto but that's not what I want either.

However, member All Taken recently put me onto the Highpoint 2720SGL which offers an 8x interface with native OS X support. The only problem with it was that it wasn't bootable in the Mac Pro (it is on a Hackintosh though). The reason this eluded me before, was the fact that Highpoint doesn't list it among their officially supported Mac products. The fact is, it works flawlessly without issue. There are no sleep or other issues I've come across after running it for a couple of weeks.

The Solution:

So while I ultimately wanted to have both my OS/Apps and Aperture Photo Library all on a single large SSD RAID0 volume, I recently decided this may never happen, so I opted to separate them.

The solution I decided on was:
  • 256GB Crucial M4 on Apricorn Velocity Solo x2 for OS/Apps
  • 3x256GB OCZ Vertex4 on the Highpoint 2720SGL for Aperture Libraries

My thinking was that my OS and Apps would benefit most from top notch small random read/write performance, which the M4 excels at. And, of course, the Aperture library (storing photos of 25-100MB in size) would benefit most from sequential read/write performance where RAID0 arrays excel. While I could have just purchased a pair of 256GB SSDs for the Aperture Library, I opted for a trio of drives to have more capacity and to really try to push performance. And, as you can imagine, that spare port on the 2720 is crying out for a fourth drive.

I had purchased the Crucial M4 some time ago when the prices came down to around $1/GB and was running it as my boot drive off the SATA2 backplane. It was trivial to connect it to the Apricorn Solo x2 and drop it in the top slot.

The rest all came together on Black Friday when the the 256GB Vertex 4 was available from Newegg for $175 CAD (it was $159 on Amazon at that time for US customers) and the 2720SGL was also on sale (and still is).

Why the Vertex 4:

I've been fairly critical of the Sandforce drives in the past, and, of course, OCZ has been one of the major vendors for Sandforce drives. On top of that, their reputation for using early adopters as beta testers is widely publicized. So why would I buy an OCZ product? Well, normally, I wouldn't for those exact reasons, however, in the case of the Vertex 4, a few things are different... (1) the drive uses a proven Marvell controller (2) the drive has been on the market for several months without any wide-spread complaints of issues (3) the performance is top notch (4) the price was rock bottom.

I did a lot of reading on the Vertex 4 and there's no MLC based drive that can write faster. There's an interesting and cool reason for this. As you may know, SLC NAND is single-bit per cell technology and it's super fast to write data to although it's very expensive. MCL NAND on the other hand, is dual-bit per cell so it's much more affordable but it takes a lot longer to write data to, hence write speeds suffer. This (and the endurance benefits) is why you see SLC NAND typically used in enterprise drives. It seem OCZ took advantage of the fact that you can treat MLC NAND somewhat like SLC NAND if you're only writing one bit per cell at any given time, and thus gain some write performance. Of course, this will only work up to 50% of the drives capacity. After which, the garbage collection needs to reorganize the data from single bit per cell to double bit per cell to free up NAND. This is pretty effective if you never write more than 50% of your free capacity at any given time. You effectively get SLC write speeds using MLC NAND when writes are not more than 50% of available free space.

While this was not a factor in my decision, I think it's a novel approach to improving performance without resorting to compression.

Additional hardware:

Besides, the Highpoint 2720 card, I also needed a couple of cables, and some way to mount the 3x Vertex 4 SSDs.

Here's the other stuff I purchased:

Installation:

The M4 on the Apricorn Solo X2...



The Vertex SSDs in their mounting bracket... note, there's room for 1 more





In the Mac Pro optical bay carrier...



The Mac Pro interior with cables running from the 2720 to the optical bay... (the idea for routing the cables comes from ConcordRules thread - Thanks!)



Everything installed...



Setting up RAID on the Highpoint 2720:

OS X recognizes the 2720 as a Parallel SCSI/SAS controller...



Drives will show up as SCSI/eSATA drives with an orange icon. You can manage them in Disk Utility, just like any other drive. And therefore you have all the software RAID options at your disposal that OS X natively offers.



If you want to take advantage of other RAID modes the 2720 offers, you can do so... you simply need to install their web RAID management interface (for the 2722) to allow you to access the card's firmware through a browser. I tried this and it works fine. However, it's RAID0 stripe size options were limited so one option (32K IIRC) so after playing around with it, I rebuilt my final RAID array using disk utility. More on performance and stripe size options coming up.

PCIe Slot:

All the recent Mac Pro's have a pair of 4x (electrical) slots at the top that connect through the ICH and a pair of 16x slots at the bottom connected to the X58 IOH.

I believe the top two x4 slots are switched... in other words, even though cards in both those slots will negotiate a x4 connection, they will be sharing that x4 (2GB/s) bandwidth. At any rate, what I found was even more damning... it appears there's something in the architecture of the Mac Pro that's limiting bandwidth of those top slots to 1GB/s or about half what I expected.

The 2720 with 3x SSDs in one of the top slots could not sustain transfers more than about 1000MB/s. In one of the x16 slots, I could get sustained transfer rates around 1370MB/s which is more in line with the limit of the three drives in RAID0.

Since I have a pair of recent Apple displays with mini display port connectors that I'm driving with a pair of GT120's, I'm currently running the Highpoint in one of the crippled x4 slots but I hope to rectify that soon once I get a single slot graphics card with dual MDP ports.

Drive benchmarking tools:

These are the drive benchmarking tools that I'm familiar with:
  • Xbench – (free) last updated in 2006 before SSDs and notoriously inconsistent
  • AJA – (free) sequential read/write tests only and doesn’t appear to work on boot volumes
  • Blackmagic – (free in the app store) sequential read/write tests only
  • Disktester – ($40) No comment
  • Quickbench – ($15) offers sequential and random tests for a reasonable price

I purchased Quickbench. It seemed like a small investment compared to the hardware and allowed me to test a number of things to greater depth than any of the free tools. Having said that, the sequential speeds reported by AJA, Blackmagic, and Quickbench are all consistent which is a good sign and a solid endorsement for the free tools if that’s all you need to test.

Optimal stripe size:

I've always subscribed to the school of thought that stripe size should be a function of the type of data you're storing/accessing... a small stripe size for smaller files... and larger stripe sizes for larger files. When SSDs came along, the write amplification penalty due to large NAND page and block sizes also supported the use of larger stripe sizes. So when I started into this project, I was fully expecting to run as large a stripe size as I could.

In my research however, I found another guy that had been testing various RAID0 stripe sizes with the Vertex 4 (on a PC) and found that they performed better with smaller strip sizes... right down to 4KB. I decided to do a bunch of stripe size testing of my own before committing to one on my array.

Using the x16 slot for my testing, I tried 128K, 64K, 32K and 16K (the smallest OS X RAID0 allows) and the results were consistently better with decreasing stripe size. I forgot to take a screen shot of the benchmark tool on the larger stripes, but if I recall correctly, the differences were significant between 128K and 16K for larger sequential transfers (on the order of around 100MB/s but still less than 10%) and insignificant at lower 4k transfer sizes.

OS X vs Highpoint RAID0:

I compared RAID0 performance using the 2720 firmware vs. Disk Utility in OS X and results were identical in both scenarios.

Single vs Dual vs Triple SSDs in RAID0:

I also opted to test the Vertex 4 and 2720's ability to scale as drives were added to the array. Here are the max sequential read / write speeds in each case:
Single Drive: 489MB/s / 465MB/s
Dual Drive RAID0: 956MB/s / 927MB/s
Triple Drive RAID0: 1066 MB/s 1370 MB/s

Note that the read speeds don't scale well linearly beyond two drives. I'm not sure if this is the card or the drives. However write speeds do scale well as drive count increases (at least up to 3 drives).

Final Configuration:

For simplicity, I opted to setup the final 3x 256GB RAID0 array in disk utility with a 16K stripe.

Here’s the benchmarks: (attached)
Attached Thumbnails
Click image for larger version

Name:	Final Screen Shot 2012-11-29 at 10.00.43 PM.png
Views:	125
Size:	157.5 KB
ID:	382391   Click image for larger version

Name:	Final Screen Shot 2012-11-29 at 10.00.14 PM.png
Views:	94
Size:	156.6 KB
ID:	382392   Click image for larger version

Name:	Final Screen Shot 2012-11-29 at 10.01.39 PM.png
Views:	96
Size:	167.2 KB
ID:	382393  

__________________
tools: nMP for photography, rMBP for working, iPad for surfing, iPhone for communicating, Mac Mini for entertaining
Canon tools: 5D Mark III 24-105L/70-300L/35L/50L/85L for capturing
VirtualRain is offline   5 Reply With Quote
Old Dec 8, 2012, 07:32 PM   #2
All Taken
macrumors 6502a
 
Join Date: Dec 2009
Location: UK
Nice post dude, you see that area next to the CPU heatsink? The large area that looks more than comfortable for an additional 4x SSD's?

That area or the use of a DX4 will make full use of the card. If you decide to put some by the heat sink on the empty daughterboard space be sure to place a piece of plastic under the SSD's.

When you route the cables you could unscrew the two screws securing the PCI fan and route the cables behind and under it.

Last edited by All Taken; Dec 8, 2012 at 07:40 PM.
All Taken is offline   0 Reply With Quote
Old Dec 8, 2012, 07:42 PM   #3
hfg
macrumors 68020
 
hfg's Avatar
 
Join Date: Dec 2006
Location: Cedar Rapids, IA.
Very nice writeup ... thanks for sharing!

To make the AJA work on boot drives, use "Get Info" to add yourself to the list at the bottom, with read/write permissions, and click the "gear" and make yourself the owner.

Keep us posted on your progress with this project.




-howard
hfg is offline   0 Reply With Quote
Old Dec 8, 2012, 07:55 PM   #4
mokeiko
macrumors 6502
 
Join Date: Jul 2007
Very great information and a really nice rig you got there (thanks for the photos as well). I did something very similar except I used 4 SSD's in the optical bay. I created a RAID 0 on two SSD's each. I used the same Highpoint cable (routed the same way you have shown) and power splitters. I didn't have a need for an optical DVD drive. The inspiration came from Concord Rules as well.

mokeiko
mokeiko is offline   0 Reply With Quote
Old Dec 9, 2012, 03:19 AM   #5
VirtualRain
Thread Starter
macrumors 603
 
Join Date: Aug 2008
Location: Vancouver, BC
Thanks guys. It was a fun project and should keep the upgrade itch away now until a new Mac Pro reveals itself.
__________________
tools: nMP for photography, rMBP for working, iPad for surfing, iPhone for communicating, Mac Mini for entertaining
Canon tools: 5D Mark III 24-105L/70-300L/35L/50L/85L for capturing
VirtualRain is offline   0 Reply With Quote
Old Dec 10, 2012, 08:49 AM   #6
odinsride
macrumors 65816
 
odinsride's Avatar
 
Join Date: Apr 2007
Great writeup, I love reading things like this
__________________
"It is just that heavy metal musicians write in minor keys, and when you do that, you frighten people." - Ronnie James Dio (RIP)
Flickr
odinsride is offline   0 Reply With Quote
Old Dec 10, 2012, 08:12 PM   #7
VirtualRain
Thread Starter
macrumors 603
 
Join Date: Aug 2008
Location: Vancouver, BC
BTW, I'm interested in knowing if anyone else has hit up against a 1GB/s bottleneck using one of the top x4 PCIe slots in a 2009+ Mac Pro.
__________________
tools: nMP for photography, rMBP for working, iPad for surfing, iPhone for communicating, Mac Mini for entertaining
Canon tools: 5D Mark III 24-105L/70-300L/35L/50L/85L for capturing
VirtualRain is offline   0 Reply With Quote
Old Jan 19, 2013, 01:11 PM   #8
gerardomare1
macrumors newbie
 
Join Date: Jan 2013
Bootable?

Hello VirtualRain
ITs the raid configuration bootable?
i want put only the 2720SGL with 2 sad
thanks a lot for you attention. Sorry for my english
gerardomare1 is offline   0 Reply With Quote
Old Jan 19, 2013, 02:32 PM   #9
VirtualRain
Thread Starter
macrumors 603
 
Join Date: Aug 2008
Location: Vancouver, BC
No, the Highpoint is not bootable.
__________________
tools: nMP for photography, rMBP for working, iPad for surfing, iPhone for communicating, Mac Mini for entertaining
Canon tools: 5D Mark III 24-105L/70-300L/35L/50L/85L for capturing
VirtualRain is offline   0 Reply With Quote
Old Jan 19, 2013, 03:23 PM   #10
wonderspark
macrumors 68030
 
wonderspark's Avatar
 
Join Date: Feb 2010
Location: Denver, CO
Quote:
Originally Posted by VirtualRain View Post
(Cool setup)
The four SSDs in the ODD bay... do you have two of the SATA power splitters, sending two to a pair of SSDs, and the other two to one SSD and the optical drive?

----------

And, do they give you any trouble, power-wise?
__________________
Wait a second... So you're telling me anything that happens in the sky is legal, and there's a giant crime-blimp flying around this place? I don't know how I missed that.
wonderspark is offline   0 Reply With Quote
Old Jan 19, 2013, 03:43 PM   #11
gerardomare1
macrumors newbie
 
Join Date: Jan 2013
thanks a lot
gerardomare1 is offline   0 Reply With Quote
Old Jan 19, 2013, 11:19 PM   #12
VirtualRain
Thread Starter
macrumors 603
 
Join Date: Aug 2008
Location: Vancouver, BC
Quote:
Originally Posted by wonderspark View Post
The four SSDs in the ODD bay... do you have two of the SATA power splitters, sending two to a pair of SSDs, and the other two to one SSD and the optical drive?

----------

And, do they give you any trouble, power-wise?
I have three SSDs in the optical bay and just one of the aforementioned 1 to 3 SATA power splitters. And no, they don't give me any trouble.
__________________
tools: nMP for photography, rMBP for working, iPad for surfing, iPhone for communicating, Mac Mini for entertaining
Canon tools: 5D Mark III 24-105L/70-300L/35L/50L/85L for capturing
VirtualRain is offline   0 Reply With Quote
Old Jan 20, 2013, 12:51 AM   #13
wonderspark
macrumors 68030
 
wonderspark's Avatar
 
Join Date: Feb 2010
Location: Denver, CO
Quote:
Originally Posted by VirtualRain View Post
I have three SSDs in the optical bay and just one of the aforementioned 1 to 3 SATA power splitters. And no, they don't give me any trouble.
Ah, yes. I forgot that thing splits into three.

I'm figuring that the Superdrive pulled about 30 Watts from one of the plugs, as it says 5v/12v === 2.0A/2.5A on the unit itself. Assuming the 12v/2.5A is the power draw, that equals 30W. The Samsung 830 SSD draws 0.24W, and the new 840 Pro SSD draws only 0.068W during active use, so even using the less efficient 830, 30 Watts would cover 120 SSDs, so I'm sure four of them on a splitter is totally insignificant.

This leads me to imagine replacing my 8-bay 3.5" HDD RAID box with 12 SSDs inside the ODD bay instead, all connected to my 12 internal ports on my Areca RAID card in RAID 6. I think it would probably saturate the x8 lane RAID card, haha! Too bad we're not in the days of cheap 1TB SSDs.

----------

Looking around, I'm seeing all kinds of variable reports on power consumption, but none are more than a half Watt.
__________________
Wait a second... So you're telling me anything that happens in the sky is legal, and there's a giant crime-blimp flying around this place? I don't know how I missed that.
wonderspark is offline   0 Reply With Quote
Old Jan 20, 2013, 04:44 AM   #14
AjeeB
macrumors member
 
Join Date: Mar 2010
anybody can point me to get long sata cable to reach ! sata3 controller ?
__________________
Mac Pro 3,1: 2x2.8Ghz 8core + 8GB RAM + Radeon HD 6870 + 4TB + LG W2443T (FHD),FusionDrive
iPhone5-32gb ,ATV3
AjeeB is offline   0 Reply With Quote
Old Jan 20, 2013, 05:29 PM   #15
VirtualRain
Thread Starter
macrumors 603
 
Join Date: Aug 2008
Location: Vancouver, BC
Quote:
Originally Posted by AjeeB View Post
anybody can point me to get long sata cable to reach ! sata3 controller ?
Look on Newegg for SATA cables 1m in length. There appears to be plenty to choose from.
__________________
tools: nMP for photography, rMBP for working, iPad for surfing, iPhone for communicating, Mac Mini for entertaining
Canon tools: 5D Mark III 24-105L/70-300L/35L/50L/85L for capturing
VirtualRain is offline   0 Reply With Quote
Old Jan 20, 2013, 10:57 PM   #16
Tucom
macrumors 6502a
 
Join Date: Jul 2006
Is that ECC RAM? There's no heatsinks on the RAM
Tucom is offline   0 Reply With Quote
Old Jan 21, 2013, 02:29 AM   #17
VirtualRain
Thread Starter
macrumors 603
 
Join Date: Aug 2008
Location: Vancouver, BC
Quote:
Originally Posted by Tucom View Post
Is that ECC RAM? There's no heatsinks on the RAM
Yes it's ECC sticks from OWC. I'm surprised you're surprised by the lack of heat spreaders. Stock RAM doesn't include heat spreaders either. Heat spreaders on RAM are more about cosmetic bling and marketing than cooling.
__________________
tools: nMP for photography, rMBP for working, iPad for surfing, iPhone for communicating, Mac Mini for entertaining
Canon tools: 5D Mark III 24-105L/70-300L/35L/50L/85L for capturing
VirtualRain is offline   0 Reply With Quote
Old Jan 21, 2013, 06:46 AM   #18
Concorde Rules
macrumors 68000
 
Join Date: Sep 2007
Just for the record, AJA System Test will let you bench a boot drive (See my thread in fact!).

You just need to change the boot disk permissions to allow anything to write to it.
Concorde Rules is offline   0 Reply With Quote
Old Jan 21, 2013, 09:50 AM   #19
AjeeB
macrumors member
 
Join Date: Mar 2010
Quote:
Originally Posted by VirtualRain View Post
Look on Newegg for SATA cables 1m in length. There appears to be plenty to choose from.
thanks friend ,
but what about the black combo cable there connected to SATA3-card ? is it the Apple(bundled) or you buy it ?
gr8 system you got there , cheers ! nice work
__________________
Mac Pro 3,1: 2x2.8Ghz 8core + 8GB RAM + Radeon HD 6870 + 4TB + LG W2443T (FHD),FusionDrive
iPhone5-32gb ,ATV3
AjeeB is offline   0 Reply With Quote
Old Jan 21, 2013, 10:22 AM   #20
DPUser
macrumors 6502
 
Join Date: Jan 2012
Quote:
Originally Posted by AjeeB View Post
thanks friend ,
but what about the black combo cable there connected to SATA3-card ? is it the Apple(bundled) or you buy it ?
gr8 system you got there , cheers ! nice work
The combo cable is terminated in two separate (Power and SATA) connectors at the mother board, I used the power portion of the combo cable only, splitting it at the optical bay using a 1 to 3 splitter adaptor. I disconnected the SATA connection of the combo cable from the mother board and ran a standard SATA cable from the mother board to the bay. Additional SATA comes from the PCIe card.
DPUser is offline   0 Reply With Quote
Old Jan 21, 2013, 11:41 AM   #21
AjeeB
macrumors member
 
Join Date: Mar 2010
got it thanks ! you rock !
__________________
Mac Pro 3,1: 2x2.8Ghz 8core + 8GB RAM + Radeon HD 6870 + 4TB + LG W2443T (FHD),FusionDrive
iPhone5-32gb ,ATV3
AjeeB is offline   0 Reply With Quote
Old Feb 14, 2013, 02:52 AM   #22
Chumnick
macrumors newbie
 
Join Date: Dec 2011
Hello VirtualRain!
Tell me please, with Apricorn Velocity Solo x2 in system info-hardware-serial-ata, what series chipset shows: unknown or other? Is bootable in Mac OS and Windows? And what version AHCI supported?
Thanks!
Chumnick is offline   0 Reply With Quote
Old Feb 14, 2013, 11:29 PM   #23
VirtualRain
Thread Starter
macrumors 603
 
Join Date: Aug 2008
Location: Vancouver, BC
Quote:
Originally Posted by Chumnick View Post
Hello VirtualRain!
Tell me please, with Apricorn Velocity Solo x2 in system info-hardware-serial-ata, what series chipset shows: unknown or other? Is bootable in Mac OS and Windows? And what version AHCI supported?
Thanks!
The Solo x2 boots natively in OS X. I can't comment on how it behaves in Windows... I despise Windows and will never let that sad excuse for an OS ever reside on any of my systems... ever again! I'd be completely remiss if I didn't aggressively discourage you from using Windows of any kind on your Mac!

Here's how it appears in Sys Info...
Attached Thumbnails
Click image for larger version

Name:	Screen Shot 2013-02-14 at 9.30.10 PM.png
Views:	48
Size:	137.6 KB
ID:	397065  
__________________
tools: nMP for photography, rMBP for working, iPad for surfing, iPhone for communicating, Mac Mini for entertaining
Canon tools: 5D Mark III 24-105L/70-300L/35L/50L/85L for capturing
VirtualRain is offline   0 Reply With Quote

Reply
MacRumors Forums > Apple Hardware > Desktops > Mac Pro

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 06:15 AM.

Mac Rumors | Mac | iPhone | iPhone Game Reviews | iPhone Apps

Mobile Version | Fixed | Fluid | Fluid HD
Copyright 2002-2013, MacRumors.com, LLC