Pegasus2 - is this too slow?

[joker00]

I just got the P2, and have 4x1TB drives (Seagate Barracuda ST1000DM003) set up as Raid 5.

This is on the Thunderbolt2 connection.

I see about 438MB/s Write and 210-220 MB/s Read.

Do these numbers sound right? I think they are slower then my esata box that used the same drives (yes they where reformatted when placed in the P2 R4 box).

thanks for the help.

 

[chfilm]

It seems to be too slow!

Probably because of those drives...
It should be able to deliver up to 800 mb/s read speeds according to an email I got from a Promise spokesman.

 

[matoch]

It seems to be too slow!

Probably because of those drives...
It should be able to deliver up to 800 mb/s read speeds according to an email I got from a Promise spokesman.

Either that's a different unit or I'd bet that assumes raid-0.

In raid 5 I don't think 800 is anywhere near realistic unless you have some SSD's.

Mine should be here in a couple days and I can post my results.

 

[joema2]

I have the Pegasus Thunderbolt R4 (ver 1, not ver 2) in RAID 5 with 4x2TB drives, and on a 2013 iMac with 3.5Ghz i7 and 32GB RAM, get significantly higher perf. than that. In BlackMagic 2.2, with 4GB test size, about 670 MB/sec write, 516 MB/sec read. In Quickbench 4.0.5 (large), about 698 MB/sec write, 858 MB/sec read.

Drives are Toshiba DT01ACA200; 3.5", 7200 rpm 6 gbps SATA drive with 64MB internal cache. It's 8TB unformatted, about 6TB formatted for RAID 5.

https://forums.macrumors.com/threads/1691038/

My tests were done with the R4 almost empty. It might give different results with more data occupied.

Also various benchmarks behave differently. I'd suggest using several different ones and cross-checking them with a timed folder copy.

I did some testing with different stripe sizes and didn't see a huge difference. However in theory if the stripe size is roughly matched to the I/O size, perf. will be better.

However, stripe size makes a huge difference on the R4 regarding RAID 5 synchronization time; see chart in above thread for that.

 

[matoch]

https://forums.macrumors.com/threads/1691038/

I've read that post before. I remember liking it. Well done.

 

[bcuzawd]

Great news to see the Peg1 running so fast. Should see even better speeds (I hope) with the Peg2 once it's all synced-up.

 

[chfilm]

I just got the P2, and have 4x1TB drives (Seagate Barracuda ST1000DM003) set up as Raid 5.

This is on the Thunderbolt2 connection.

I see about 438MB/s Write and 210-220 MB/s Read.

Do these numbers sound right? I think they are slower then my esata box that used the same drives (yes they where reformatted when placed in the P2 R4 box).

thanks for the help.

Maybe it's still syncing

 

[slughead]

Someone might have said this already: Is it daisy-chained to the monitor or in the bottom TB2 port?

http://www.anandtech.com/show/7603/mac-pro-review-late-2013/13

Here’s where the six Thunderbolt 2 and three TB2 controllers come into play. Although you can daisy chain a 4K display onto the back of a Thunderbolt 2 storage device, doing so will severely impact available write bandwidth to that device. Remember that there’s only 20Gbps available in each direction, and running a 3840 x 2160 24bpp display at 60Hz already uses over 14Gbps of bandwidth just for display. I measured less than 4Gbps of bandwidth (~480MB/s) available for writes to a Thunderbolt 2 device downstream from the Mac Pro if it had a 4K display plugged in to it. Read performance remained untouched since display data only flows from host to display, leaving a full 20Gbps available for reads. If you’re going to connect Thunderbolt 2 devices to the Mac Pro as well as a 4K display, you’ll want to make sure that they aren’t on the same chain.

 

[joker00]

Quick Update

Thanks everyone for the interest, here's a quick update.

I called Promise. They say it's definitely to slow. I have the 1TB Barracuda drives, which are not supported. (Only the 3 & 4TBs are).

So promise told me to replace them and call back.

I have 4 new drives on order, will move all my data, again and see what happens.

I will be very surprised if it's the drives. they worked fine in my old SnartStor.

 

[joema2]

This may not be related, but I suggest you update your Promise Utility and R4 chassis firmware to the latest versions. This can be done non-destructively using the Promise Utility update option.

Also consider re-creating your RAID 5 volume with a 512k stripe size. That will require re-loading, so whether it's worth doing that before getting the new drives is up to you.

Don't go by a single benchmark program; I use Black Magic, QuickBench, XBench, and DiskTester plus do sanity tests with a timed folder copy (to/from/within) the R4, and convert that time to megabytes/sec.

Black Magic: https://itunes.apple.com/us/app/blackmagic-disk-speed-test/id425264550?mt=12
QuickBench: https://www.speedtools2.com/QuickBench.html
XBench: http://xbench.com
DiskTester: http://diglloydtools.com/disktester.html

 

[theSeb]

It seems to be too slow!

Probably because of those drives...
It should be able to deliver up to 800 mb/s read speeds according to an email I got from a Promise spokesman.

Impossible with 4x1TB drives (Seagate Barracuda ST1000DM003)

----------

This may not be related, but I suggest you update your Promise Utility and R4 chassis firmware to the latest versions. This can be done non-destructively using the Promise Utility update option.

Also consider re-creating your RAID 5 volume with a 512k stripe size. That will require re-loading, so whether it's worth doing that before getting the new drives is up to you.

Good advice. Speeds went up quite a lot with my R4 Pegasus when I upgraded the firmware from the shipped version to the latest.

 

[chfilm]

Impossible with 4x1TB drives (Seagate Barracuda ST1000DM003)

Ok maybe, they refered to the standard 4x2TB version!

 

[spaz8]

I have a promise2 R4 diskless on order.. I have to say I'm pretty disappointed with the HDD drive compatibility. The list for the Promise 1 was short, but the Promise2 list is half that size and full of HDD's that are EOL'd. With maybe 1 Enterprise model at all.

I'm a bit confused if the Diskless version only works with the WD and Hitachi drives. But if you pay the ridiculous markup for a pre-configured one you can replace drives with the seagate or Toshiba models?

I'd much rather put a Seagate Constellation ES.3 in their with 128 MB cache than the mediocre general purpose HDD"s they list. The old doc's even recommended enterprise drives for raid setups.. yet their list looks like is for a JBOD.

http://www.promise.com/media_bank/Download%20Bank/Compatibility/Pegasus2_Compatibility%20List%20v1.1-201301216.pdf

 

[joker00]

what speeds to expect

So far I only have 3 of my 4 drives.
I've tested in a raid 0.

What speeds should I expect with 3 drives in Raid 0 and Raid 5?

What speeds for 4 drives in Raid 5?

thanks.

 

[matoch]

What speeds for 4 drives in Raid 5?

thanks.

I have mine set up and synced. I have 4x3tb seagate drives. Using blackmagic I'm seeing between 550-600 MB/s write (using write back cache). And between 285 - 300 read (Using read cache). This test is with the 5GB setting within black magic.

 

[joema2]

...I'm pretty disappointed with the HDD drive compatibility. The list for the Promise 1 was short, but the Promise2 list is half that size and full of HDD's that are EOL'd....I'm a bit confused if the Diskless version only works with the WD and Hitachi drives. But if you pay the ridiculous markup for a pre-configured one you can replace drives with the seagate or Toshiba models?...

I tested three different Promise1 R4s with 4x2GB in RAID 5. The first one was pre-configured with the Toshiba DT01ACA200, firmware version MX4OABB0.

Ironically those were not on the compatibility list due to the HDD firmware being the wrong number.

The initial RAID 5 sync was very slow and Promise suspected the non-certified drives where the cause (even though they shipped them). They send me replacement HDDs, which turned out to have exactly the same firmware.

I traced the slow sync performance to using the default 128KB stripe size; it's much faster at larger sizes. The drive firmware had nothing to do with it: https://forums.macrumors.com/posts/18576138/

There is comfort in knowing there's a certified drive list. OTOH the sub-versions and firmware on HDDs change frequently. If Promise doesn't keep on top of this, you end up with an outdated list.

Ideally you'd like a "turnkey" package which was pre-tested and certified. In reality you're best served doing your own testing. I use DiskTester, which is a very robust set of drive stress tools: http://diglloydtools.com/disktester.html You'd also need to force a failure and test rebuild performance.

In general I think the Promise R4 series is a good product. It's well designed, quiet and matches Apple's design aesthetic. It is extremely fast.

 

[matoch]

I have to say I'm pretty disappointed with the HDD drive compatibility. The list for the Promise 1 was short, but the Promise2 list is half that size and full of HDD's that are EOL'd. With maybe 1 Enterprise model at all.
http://www.promise.com/media_bank/Download%20Bank/Compatibility/Pegasus2_Compatibility%20List%20v1.1-201301216.pdf

I'm not sure how much you need to worry about the compatibility list. I bought the diskless and put in 4 seagate st3000dm001 drives and it seems to be fine. Promise may not officially support it but it's working.

 

[joker00]

I'm not sure how much you need to worry about the compatibility list. I bought the diskless and put in 4 seagate st3000dm001 drives and it seems to be fine. Promise may not officially support it but it's working.

I believe that drive is the first one on Promise's compatibility list.

 

[spaz8]

I'm not sure how much you need to worry about the compatibility list. I bought the diskless and put in 4 seagate st3000dm001 drives and it seems to be fine. Promise may not officially support it but it's working.

The "old" Promise 1 R4 doc's said they recommend Enterprise drives for RAID setups. So I was trying to not go the cheapest route. I learned the hard way not to put WD greens in my Drobo.

Only the Toshiba and and 2 of the 3 Hitachi drives on the list are enterprise grade. I'd buy 4x DT01ACA200 if they still made them .. Same deal with the Ultrastars.. WD bought them so I can't source a HUA722020ALA330.

Plus there are 128 MB cache drives now I'd rather use.

The current compatibility list is pretty horrible for the Pegausus2. Doesn't even list a WD RED option, just Blue (640 gb) and Black (1 TB).

 

[matoch]

I believe that drive is the first one on Promise's compatibility list.

It was technically listed in the old document as compatible with the r4 provided it was not bought diskless.

It was not on the diskless one which I thought was ridiculous.

Either way I figure it's just a list of drives they tested and likely any sata drive will work. It's just that they will not help you if you encounter an issue with the system.

 

[joker00]

Latest Update

Ok Folks here's what I finally have:

4x 3TB drives in a RAID 5.
I'm using the Seagate Barracuda, which is on the Compatibility list.

Promise has directions for turning on ForcedReadAhead, which I did.

Using BlackMagic's Disk Speed Test I am ~600+MB/s Write Speed, and 370+ MB/s Read Speed.

I thought I'd see a bit more????

thanks.

 

[joema2]

Ok Folks here's what I finally have:

4x 3TB drives in a RAID 5.
I'm using the Seagate Barracuda, which is on the Compatibility list.

Promise has directions for turning on ForcedReadAhead, which I did.

Using BlackMagic's Disk Speed Test I am ~600+MB/s Write Speed, and 370+ MB/s Read Speed.

I thought I'd see a bit more????...

As previously posted, I consistently get about 670 MB/sec write, 516 MB/sec read. That's on a P1 R4 in RAID 5 connected to a 2013 iMac 27, i7@3.5Ghz, 32GB RAM, with disk empty. Disks are 4 x 2TB Toshiba DT01ACA200.

Those are "best case" numbers; I don't really like BlackMagic because of the fluctuation. If possible I suggest you try some alternate benchmarks which give averaged numbers, like QuickBench.

I'm using 512k KB stripe size, although limited testing didn't show a huge I/O perf. difference in various stripe sizes. Other settings from Promise Utility:

Global Physical Drive Settings: Enable write cache, enable read look ahead cache, enable command queuing.

Disk Array: enable media patrol, enable PDM

Logical Drive Cache Cache Policy: Read Ahead, Write Back.

Controller: hardware rev. A2, host driver version: 5.1.66, BIOS ver: 5.02.0000.98

Controller Advanced Info: write back cache flush interval: 3 sec., encloser polling interval: 15 sec, forced read ahead: disabled, memory size: 512MB, write through mode: disabled, adaptive writeback cache: disabled.

Note I'm not using ForcedReadAhead, I wonder if that's a factor?

 

[joema2]

OK I did more testing with ForcedReadAhead enabled on my P1 R4. Note this requires using the Promise command-line utility from Terminal, as the Promise GUI doesn't work. Details: http://kb.promise.com/KnowledgebaseArticle10394.aspx?Keywords=force+readahead+pegasus

Note also you should *disable* Spotlight indexing for any drive you're testing. Spotlight will perturb the results and you can't trust them. To disable Spotlight for those drives, System Preferences->Spotlight-Privacy, and drag the drives there. When done testing, just remove them and indexing will resume.

With ForcedReadAhead enabled, I couldn't see a significant difference on BlackMagic, but it's hard to say since it fluctuates so much. I also couldn't see any difference on QuickBench, but it did improved some other benchmarks, inc'l digLloydTools DiskTester ReadFiles and IOStreams. It had no apparent write penalty on these tests.

I also did various timed copy tests to/from the R4 using Finder, but couldn't see much difference with ReadAhead on or off. There can be a significant difference with Spotlight on or off.

Based on these tests I'll leave ForcedReadAhead enabled on my P1 R4. I suggest you get DiskTester -- it's a professional level disk test tool, and re-evaluate your tests with Spotlight turned off.

 

[joker00]

Update with DiskTester

thanks joema2 for your help.

I got disk tester and got the following info:
————————— Averages for “RAID” (2GB/4MB, 5 iterations) ————————
Iteration Write MB/sec Read MB/sec
1 663 355
2 573 435
3 676 384
4 659 415
5 631 361

Slowest 573 355
Fastest 676 435

I'm not to happy with these results. This is also with spotlight turned off.

I've called promise today, got a call back, and promise's promise to look into this. no one told me the speeds are normal. I like this unit, but if they tell me to change disks one more time, I'm going to scream.

 

[joema2]

thanks joema2 for your help.

I got disk tester and got the following info:...

Please tell me the exact test and parameters you used in DiskTester. It looks like you used the "Sequential Suite" but I'm not sure. I will run the same test on my R4 so we can compare results.

At first glance it appears DiskTester's Sequential Suite uses a 32k I/O size, which is not optimal for stripe sets of 128k to 1 megabyte. It is probably adjustable but I haven't explored that yet.

Since your main concern was read performance, so far I used the "Read Files" test, also "I/O Streams". We need to use the same test with the same parameters to compare.