Thunderbolt storage real-world comparison

[analog guy]

There have been many inquiries with respect to Thunderbolt enclosures, and I've had my own questions. I finally compiled some data that others may find interesting and/or useful.

I was hoping to find a solution to house 4 drives -- 2 SSDs in RAID0 + another SSD + a fourth 3.5" drive for back-ups/archive. Ideally, it would be small, quiet, fast and inexpensive.

I measured the drive speed using BlackMagic Disk Speed Test with two Samsung 840 EVO 1TB drives in a RAID0 (AppleRAID/software) configuration. I also measured the noise from each unit as measured in dBA above the room's background noise. Each enclosure was 0.8m horizontal from my body and 0.2m below my ear.

The test system was a late 2013 iMac 27" core i7 (3.5).

For reference:
* The Sonnet Tempo Pro SSD card run inside my early 2008 Mac Pro (x16 slot) tested at 943MB/s read and 850MB/s write.
* The iMac's internal PCIe SSD (512) tested at 719MB/s read and 713MB/s write.

1: Sonnet Echo Express III-D with Sonnet Tempo Pro SSD PCI card {MSRP = $1260}
Read: 801 MB/s; Write 655 MB/s; +4dBA with both fans running, +1dBA with one fan.
Note: The Thunderbolt2 upgrade just became available. The tests were run with the TB1 card.

2: Promise Pegasus2 R4 diskless Thunderbolt2 {MSRP = $700}
Read: 823; Write 494; +3dBA

3: OWC Mercury Elite Pro Dual with both TB1 and USB3 connections {MSRP = $300}
TB Read: 381; Write 324; +3dBA
USB3 Read: 420; Write 338; +3dBA
Note: The fan on this unit is small and runs at a higher rate. I find the fan noise on this unit more objectionable than either that of the Pegasus2 or the Sonnet.

The Sonnet enclosure was far and away the fastest, but it is also the most expensive. I don't love the form factor. It isn't designed for 3.5" drives and if I want to add more drives I can do so...but I must buy additional cards. It has a lock mechanism for the TB cable. I thought it would be unnecessary but realized in this comparison that when I rotate my iMac a light tug on the iMac end of the cable sometimes resulted in disconnecting the cable from the other enclosures.

The Pegasus2 allows one to use 4 drives. I was hoping to love it but was a bit disappointed in the performance. I'd like to think that perhaps some of the lower performance was due to the lack of TB2 on the iMac, but a 2-drive RAID0 is under the TB1 bandwidth limit, and clearly the Sonnet enclosure can squeeze more out of the drives with TB1. It can handle 2.5 and 3.5" drives.

I like the size and price of the OWC enclosure, and the fact that it can use USB has some appeal. Unfortunately, the TB performance was quite disappointing -- slower than USB3? It is not upgradeable to TB2. The small form factor is great, but one must use an external power supply, which takes away from the form factor.

Hope this helps someone. I'm not quite sure what I will do but after this test I'm leaning toward the Sonnet setup. It is substantially more expensive than other options, but now that it is here...

----------

One additional point regarding sleep & wake:

1: Sleep is immediate & wake takes a second or two with the Sonnet but both work as expected. The Enclosure does power down when the Mac is asleep.

2: The Pegasus2 does sleep and wake (& power down), but it takes about 20 seconds or so before the drive(s) is recognized by the OS. This is a noticeable lag upon wake from sleep.

3: The OWC enclosure allows immediate sleep & wake (much better than the other two) but does not power down during sleep.

 

[AppleDApp]

I was looking to get the OWC enclosure. what do you think caused the USB speeds to be faster then the TB speeds?

Any idea why it is so much slower than the other enclosure? Different thunderbolt controllers?

 

[ender78]

Those TB results from the OWC enclosure are VERY disappointing. I would expect 800MB/sec plus from two 840s.

 

[analog guy]

I was looking to get the OWC enclosure. what do you think caused the USB speeds to be faster then the TB speeds?

Any idea why it is so much slower than the other enclosure? Different thunderbolt controllers?

i'm not sure why the USB speeds would be faster. the USB3 speeds seem to be about what one might expect, but it suggests something wrong with the TB speeds. whether that is the controller or something else (what could it be?) i cannot say.

Those TB results from the OWC enclosure are VERY disappointing. I would expect 800MB/sec plus from two 840s.

i agree.

i was excited by the prospect of it. i guess it is fine for two 3.5" drives, but it would always have that limitation.

guess i'm in the hunt for a single 3.5" drive enclosure (USB3 might be fine) for some back-ups if i decide not to keep the r4.

oh, it is important to note that both the r4 and OWC enclosure offer hardware RAID. (well, with the OWC you select RAID0, RAID1, JBOD or independent mode via dial on the back.) not sure if there would be a performance difference, though with today's processors i'd imagine the overhead with a software RAID is not a huge del.

 

[AppleDApp]

I guess i'll be looking for a 4 bay USB3 enclosure. However I do like the daisy chain capabilities.

 

[Mr. Retrofire]

Those TB results from the OWC enclosure are VERY disappointing. I would expect 800MB/sec plus from two 840s.

Yeah, disappointing results. I use the OWC TB1/USB 3.0 enclosure as a “bridge” solution. The fast TB1 connection is compatible with my Early-2011 17" MBP and compatible with a Haswell/Broadwell Mac mini.

 

[FredT2]

3: OWC Mercury Elite Pro Dual with both TB1 and USB3 connections {MSRP = $300}
TB Read: 381; Write 324; +3dBA
USB3 Read: 420; Write 338; +3dBA
Note: The fan on this unit is small and runs at a higher rate. I find the fan noise on this unit more objectionable than either that of the Pegasus2 or the Sonnet.

I like the size and price of the OWC enclosure, and the fact that it can use USB has some appeal. Unfortunately, the TB performance was quite disappointing -- slower than USB3? It is not upgradeable to TB2. The small form factor is great, but one must use an external power supply, which takes away from the form factor.

3: The OWC enclosure allows immediate sleep & wake (much better than the other two) but does not power down during sleep.

Thanks for this information. The OWC was at the top of my list, but it's off now. Slower, more noise, and no power down during sleep is not good.

Have you looked at the Caldigit T3?

 

[analog guy]

Thanks for this information. The OWC was at the top of my list, but it's off now. Slower, more noise, and no power down during sleep is not good.

Have you looked at the Caldigit T3?

i compiled a list of about 10 options. the caldigit t3 was on that list but did not make the top 3 at the time.

the reasons were:
- the pegasus2 diskless costs less (i don't need the drives in the caldigit) and holds a 4th drive
- the pegasus2 and sonnet are tb2
- i have a feeling (just a hunch) that the caldigit will be as noisy or noisier than the r4 and mores than the sonnet

i do see that they are advertising speeds of 850MB/s for 3xSSDs for the cal digit.

on the other hand, i saw at least one review that suggested blazing speeds for the pegasus2 with four samsung 840 EVO 1TB drives. (http://www.tekrevue.com/thunderbolt-2-benchmarks-promise-pegasus2-r4-four-samsung-evo-ssds/) i don't understand why the 2-drive performance i saw with the same drives would be SO much slower.

i don't currently have four 1TB SSDs to allocate to a test, and even if i did my plan was not to run a 4-drive SSD.

any ideas?

 

[VirtualRain]

Dude, nice review but A couple of things...

1. I would ditch BlackMagic... It does ok with single disks but can be erratic or an absolute fail with RAID0 and SSDs in my experience. It doesn't even work properly for me consistently on the nMP SSD which uses a Samsung controller. Spend $15 and get Quickbench... It supports a broad range of sequential and random I/O test patterns. It's well worth it when evaluating a series of $1200 setups. For example, someone last year used it to evaluate an Accelsior and found it under-performed the Sonnet at Random I/O... Something BlackMagic could not reveal.

2. If you have time and inclination, I would try a few different stripe sizes for the RAID0 array with the Pegasus2 and Sonnet... Those drives/enclosures might behave very differently (or not) with different stripe sizes. It's worth trying 16k and 128k to see if there's any difference (but only if you ditch BlackMagic)

3. If your results still stand after some of this additional testing, I would reach out to Promise and see if this is consistent with their expectations. The write speeds you're getting are inexplicable.

EDIT: this is from the review you're comparing to...

Note what a difference the block size makes in this testing... Who knows what BlackMagic is using.

If you don't like Quickbench, maybe the DiskTest tool they used is another option?

Oh, one last thought... I would bench (with QuickBench) a single drive in each enclosure to see how linear the scaling is.

 

[analog guy]

Dude, nice review but A couple of things...

1. I would ditch BlackMagic... It does ok with single disks but can be erratic or an absolute fail with RAID0 and SSDs in my experience. It doesn't even work properly for me consistently on the nMP SSD which uses a Samsung controller. Spend $15 and get Quickbench... It supports a broad range of sequential and random I/O test patterns. It's well worth it when evaluating a series of $1200 setups. For example, someone last year used it to evaluate an Accelsior and found it under-performed the Sonnet at Random I/O... Something BlackMagic could not reveal.

2. If you have time and inclination, I would try a few different stripe sizes for the RAID0 array with the Pegasus2 and Sonnet... Those drives/enclosures might behave very differently (or not) with different stripe sizes. It's worth trying 16k and 128k to see if there's any difference (but only if you ditch BlackMagic)

3. If your results still stand after some of this additional testing, I would reach out to Promise and see if this is consistent with their expectations. The write speeds you're getting are inexplicable.

will test with quick bench. i'd tested with some other utils and they were directionally in agreement with blackmagic, thus i didn't post. i'll see if things are different with QB.

i did test a few stripe sizes when i set up the RAID0, but that was done when the drives were on the Tempo Pro card within a MP3,1.i figured results would transfer to whatever enclosure i used. is that wrong?

on a slight tangent, is there anything these days that makes a HW RAID0 (or RAID10) faster than a SW RAID? we have so much CPU power that it seems like for basic RAID setups SW RAID is not a draw-back, and HW RAIDs can be incompatible if one ever needs to switch enclosures.

ps these drives are samsung 840 EVOs. i didn't contact promise as i suspect i'll get a 'not supported' answer.

 

[VirtualRain]

will test with quick bench. i'd tested with some other utils and they were directionally in agreement with blackmagic, thus i didn't post. i'll see if things are different with QB.

i did test a few stripe sizes when i set up the RAID0, but that was done when the drives were on the Tempo Pro card within a MP3,1.i figured results would transfer to whatever enclosure i used. is that wrong?

on a slight tangent, is there anything these days that makes a HW RAID0 (or RAID10) faster than a SW RAID? we have so much CPU power that it seems like for basic RAID setups SW RAID is not a draw-back, and HW RAIDs can be incompatible if one ever needs to switch enclosures.

ps these drives are samsung 840 EVOs. i didn't contact promise as i suspect i'll get a 'not supported' answer.

Maybe you'll prove me wrong, but I've found QuickBench more accurate and reliable than BM.

It's possible the enclosure SATA to PCIe silicon may perform better/worse with different stripe sizes. But probably not enough to make much difference. So forget that suggestion given you've already tested different stripes with the drives.

If you can enable HW RAID0 in the Pegasus that might be worth a comparison, the only way it might be faster is if it has some cache. You're correct that it doesn't affect CPU utilization.

You might not get an answer, but it's worth a shot. They have a lot to lose if a few reviews like yours show crappy write performance, and I know there is some kind of Promise rep that has an account here on MR so we might even get a reply from them here.

 

[VirtualRain]

The Pegasus2 allows one to use 4 drives. I was hoping to love it but was a bit disappointed in the performance. I'd like to think that perhaps some of the lower performance was due to the lack of TB2 on the iMac, but a 2-drive RAID0 is under the TB1 bandwidth limit, and clearly the Sonnet enclosure can squeeze more out of the drives with TB1. It can handle 2.5 and 3.5" drives.

.

There may be something to this (even though it's not logical)... See his comment under "Good News" at the bottom of this review about TB2 vs TB1 performance with the Pegasus2... http://www.barefeats.com/hard179.html

GOOD NEWS: When connected to the 'late 2013' Retina MacBook Pro's Thunderbolt 2.0 port, the Pegasus2 R4 with four 6Gbps SSDs performs large sequential transfers from 36% to 57% faster than when connected to the Thunderbolt 1.0 port of a 'late 2013' iMac.

 

[Dan04MCS]

How does USB 3.0 handle simultaneous reads and writes of large files?
Does it impact the rates significantly?

 

[Mr. Retrofire]

How does USB 3.0 handle simultaneous reads and writes of large files?
Does it impact the rates significantly?

Not if the bridge chip and the OS USB drivers support the USB Attached SCSI Protocol (UASP). Of course, a good quality USB 3.0 cable minimizes also the error rate. Or in other words: Even UASP is useless if you have a bad USB 3.0 cable.

 

[analog guy]

There may be something to this (even though it's not logical)... See his comment under "Good News" at the bottom of this review about TB2 vs TB1 performance with the Pegasus2... http://www.barefeats.com/hard179.html

i'll be able to perform a test on this in 24-48h (TB1 vs TB2 performance).

still, don't love the delayed wake-from-sleep on the R4.

 

[xraydoc]

Nice thread. Thanks for the great work and testing.

 

[analog guy]

TB1 vs TB2

Hey, all:
Got my TB2 upgrade card for the Sonnet Echo Express III-D enclosure and spent all of 3 minutes installing it.

Attached are the before-and-after results with a 2x840 EVO 1TB RAID0 on a Tempo Pro SSD card.

In brief (via QuickBench):
* small sequential reads/writes went from 362/319 to 433/349
* small randoms: 324/340 to 368/358
* large: 796/735 to 992/750
* extended: 840/768 to 1033/785

Blackmagic:
From 814/737 to 973/747

Attached are the detailed QB results (TB1 in this post, TB2 in the next).

 

[analog guy]

TB2 results.

 

[xraydoc]

Fast. Thanks.

 

[AppleDApp]

Any thoughts on the ThunderBay IV?

 

[analog guy]

Any thoughts on the ThunderBay IV?

Image

i have not tested it. 1 month ago it would have been quite appealing to me, but after seeing results i would characterize as disappointing from the 2-bay version (OWC mercury elite pro dual) i would tend to avoid it -- for use with multiple SSDs.

in those tests, i found that performance through the USB3 port was faster than the same drives through the enclosure's TB1 port.

others have reported good speeds through 2 HDDs in RAID0 configuration, but my testing suggested something is limited in the implementation. i suspect something similar *might* be true for the thunder bay iv but don't know for sure.

if all one needs is a couple bays for HDDs, it might be a good option.

 

[analog guy]

had a bit of a bizarre experience today. i was preparing to return my pegasus2 r4. my prior testing (on a 2013 iMac) revealed slower performance with a 2x840 EVO 1TB RAID0 vs my sonnet echo express iii-d.

also, there were a few performance issues that left something to be desired, including slow wake-from-sleep.

well, before returning it i figured i would test it on the nMP as i noticed increased performance when i tested the sonnet on the nMP vs iMac.

i popped in 4x1TB WD Black HDDs that i had, and i set up a hardware RAID0. i was shocked to find that quick bench reported speeds above 1000MB/s in various tests. i also tested a 3x500GB (WD Black) RAID0 and saw reads up to 1200 and writes to 1128MB/s. i was shocked.

these were HW RAIDs.

i'd previously tested the 2xSSD setup on the r4 with software RAID0 (as they are used in the sonnet).

perhaps there is some type of caching going on with the r4. i didn't imagine that HW RAID0 could be so much faster than SW RAID0.

in any event, i removed the 3x500GB drives and reinserted the 4x1TB HDDs, with the intention of comparing HW to SW RAID0. one by one the drives all switched to red status. 3 were reported as dead and one as stale.

reinstalled the 500GB ones and now they are dead, too.

could be coincidence but it seems pretty bizarre. all were hot-swapped, which is allowed by the r4.

made me not want to insert my SSDs for testing another time.

when i saw those HDD results i was rethinking my decision to return the unit -- i really like the promise RAID utility, too. after these failures i'm pretty sure it's going back. could be the unit, could be something else.

bizarre to have SEVEN drive failures in short order.

guess i'll be invoking the WD Black 5-year warranty. we'll see how that works...if i can find my receipt.

 

[xraydoc]

Any thoughts on the ThunderBay IV?

Image

My opinion is this: It's not likely to be the fastest, but its price point (and likely its performance) sits between two-drive enclosures and the diskless Promise R4.

----------

had a bit of a bizarre experience today. i was preparing to return my pegasus2 r4. my prior testing (on a 2013 iMac) revealed slower performance with a 2x840 EVO 1TB RAID0 vs my sonnet echo express iii-d.

also, there were a few performance issues that left something to be desired, including slow wake-from-sleep.

well, before returning it i figured i would test it on the nMP as i noticed increased performance when i tested the sonnet on the nMP vs iMac.

i popped in 4x1TB WD Black HDDs that i had, and i set up a hardware RAID0. i was shocked to find that quick bench reported speeds above 1000MB/s in various tests. i also tested a 3x500GB (WD Black) RAID0 and saw reads up to 1200 and writes to 1128MB/s. i was shocked.

these were HW RAIDs.

i'd previously tested the 2xSSD setup on the r4 with software RAID0 (as they are used in the sonnet).

perhaps there is some type of caching going on with the r4. i didn't imagine that HW RAID0 could be so much faster than SW RAID0.

in any event, i removed the 3x500GB drives and reinserted the 4x1TB HDDs, with the intention of comparing HW to SW RAID0. one by one the drives all switched to red status. 3 were reported as dead and one as stale.

reinstalled the 500GB ones and now they are dead, too.

could be coincidence but it seems pretty bizarre. all were hot-swapped, which is allowed by the r4.

made me not want to insert my SSDs for testing another time.

when i saw those HDD results i was rethinking my decision to return the unit -- i really like the promise RAID utility, too. after these failures i'm pretty sure it's going back. could be the unit, could be something else.

bizarre to have SEVEN drive failures in short order.

guess i'll be invoking the WD Black 5-year warranty. we'll see how that works...if i can find my receipt.

Are you certain the drives are actually dead and not just corrupted? Do you have something with which to format them one by one on the Mac, then retry them in the Promise?

 

[chfilm]

It pretty much sounds like they're just corrupted to me! I'd use a usb docking station and re initialize the drives!

 

[analog guy]

My opinion is this: It's not likely to be the fastest, but its price point (and likely its performance) sits between two-drive enclosures and the diskless Promise R4.

----------



Are you certain the drives are actually dead and not just corrupted? Do you have something with which to format them one by one on the Mac, then retry them in the Promise?

it's a good thought.

if simply corrupted, why would they show as dead *and* be inaccessible (e.g., impossible to delete the RAID and reformat them)?

i thought the same but have been unable to access them in any way now.