Software RAID using JBOD enclosure?

[hieronymuspool]

Hi all,

I'm considering setting up a RAID0 system, using 3 / 4 disks, through a combination of a JBOD enclosure and software RAID, either MacOS's own software or SoftRAID.

Would any of you have insights concerning the following?

1 Am I correct to I believe you can use any JBOD enclosure, without any RAID hardware controllers built in, for RAID? That is, can and will the disks inside be seen by the OS as separate disks? Or are there certain JBOD enclosures that "force" its disks to be seen as a single big disk, rendering such an enclosure unable to be set up into a software RAID?

2 What are the differences between a JBOD enclosure and a RAID enclosure, apart from the RAID hardware controller in the RAID enclosure? F.i. are the cooling systems a significant difference between the two?

Thanks for your insights.

[background info]
Considering this RAID0 setup for my video editing needs. I'll be using this RAID0 as a fast disk for source / proxy footage, to edit from. Its footage (RAW / proxy) will be properly backed up on an archival disk. NLE project files, media cache and exports will reside on / got to another disk.

 

[casperes1996]

1 Am I correct to I believe you can use any JBOD enclosure, without any RAID hardware controllers built in, for RAID? That is, can and will the disks inside be seen by the OS as separate disks? Or are there certain JBOD enclosures that "force" its disks to be seen as a single big disk, rendering such an enclosure unable to be set up into a software RAID?

Well, if there's no chip in the enclosure that logically combines the disks, the OS should see them all independently, yes.

2 What are the differences between a JBOD enclosure and a RAID enclosure, apart from the RAID hardware controller in the RAID enclosure? F.i. are the cooling systems a significant difference between the two?

Aside from variations between individual enclosures, I can't think of any intrinsic differences aside from the RAID chips. Cooling obviously varies on a case by case basis whether it's RAID or not, as does a lot of other factors such as build quality.

 

[ColdCase]

1) Only JBOD enclosures that have a hardware RAID function can concatenate disks. I think only a couple relatively inexpensive two disk models may force you to use RAID.

2) RAID enclosures are more money and have a RAID chip, otherwise near identical.

If you don't need a lot of storage space in a single volume, I found a SSD will provide much better performance than a bunch of rotational disks raided to together. I have some archive footage on a very fast Raid with fast rotational drives and I cringe every time I need to use some of that footage.. and its not even HD. SSDs to 2TB seem to be relatively reasonable priced now days, but you can get much larger.

 

[bopajuice]

What enclosures are you looking at. I’m curious about this as well.

I was using a NAS, but I also leave my Mac Mini on all the time. Wondering if just using an enclosure connected directly to the Mini would have any benefits.

 

[casperes1996]

1) Only JBOD enclosures that have a hardware RAID function can concatenate disks. I think only a couple relatively inexpensive two disk models may force you to use RAID.

2) RAID enclosures are more money and have a RAID chip, otherwise near identical.

If you don't need a lot of storage space in a single volume, I found a SSD will provide much better performance than a bunch of rotational disks raided to together. I have some archive footage on a very fast Raid with fast rotational drives and I cringe every time I need to use some of that footage.. and its not even HD. SSDs to 2TB seem to be relatively reasonable priced now days, but you can get much larger.

I've got a SanDisk Extreme 510 external SSD. I can highly recommend it for sequential data over USB, though admittedly the random accesses are not quite as high as I would've hoped - then again, if you're looking at RAID with spinners, random IO is hardly your concern, so yeah. SSDs are a good bet.

Of course SSDs usually also give you great random IO on top

 

[hieronymuspool]

Thanks for your feedback, casperes1996 and ColdCase. It seems like any JBOD enclosure would do the trick for software RAID then.

Coldcase, I definitely see your point. However, apart from smaller and short-term commercial projects, I also work on long-term documentary films with potentially dozens, if not hundreds of hours of material, easily leading to multiple TB's even in ProRes Proxy. For me it's therefore all about the combination speed and disk space, hence RAID using HDD's. The HDD's I'm after I at least select based on their sequential read speeds - since they'll almost exclusively will be read from when editing in my NLE. For the project files, media cache and exports I am considering an SSD however.

bopajuice, looking into these JBOD enclosures:

ICY BOX IB-3640SU3
&
Fantec QB-35US3-6G. Judging by the looks of the enclosure, I suspect you could also look into the brands Startech and Mediasonic - they seem to be the same products, just a different brand name for a different market.

You can check out other models in these series for 2 bay, or with hardware RAID.

 

[casperes1996]

Coldcase, I definitely see your point. However, apart from smaller and short-term commercial projects, I also work on long-term documentary films with potentially dozens, if not hundreds of hours of material, easily leading to multiple TB's even in ProRes Proxy. For me it's therefore all about the combination speed and disk space, hence RAID using HDD's. The HDD's I'm after I at least select based on their sequential read speeds - since they'll almost exclusively will be read from when editing in my NLE. For the project files, media cache and exports I am considering an SSD however.

Sorry if I've missed it and you've already said it, but what NLE are you using? FCP X? Premiere? Avid Media Composer?

My expertise is with Final Cut, and for that at least, your plan sounds perfect. Store media on fast sequential storage and your Library file on fast random IO since its written to with every single change.

Perhaps getting hardware RAID may be better for you though. Software RAID is more error-prone which seems bad for your use case
Then again, you might not actually lose anything if original media as well as library files are kept elsewhere and the RAID is only for proy data - then you just remake the proxies and link them to the source and nothing's lost. Anyhow, I'm off on a tangent now

 

[hieronymuspool]

Sorry if I've missed it and you've already said it, but what NLE are you using? FCP X? Premiere? Avid Media Composer?

My expertise is with Final Cut, and for that at least, your plan sounds perfect. Store media on fast sequential storage and your Library file on fast random IO since its written to with every single change.

Perhaps getting hardware RAID may be better for you though. Software RAID is more error-prone which seems bad for your use case
Then again, you might not actually lose anything if original media as well as library files are kept elsewhere and the RAID is only for proy data - then you just remake the proxies and link them to the source and nothing's lost. Anyhow, I'm off on a tangent now

Cheers, I think this setup would be decent enough indeed. I've been researching this for quite a couple of days now - about time to pull the trigger.

I'm using Premiere Pro CC and a bit of Davinci Resolve 14.

I'm curious to hear why you still feel hardware RAID might be better suited for my needs?

It's a budget balancing act for me. Unless you're immediately going into the $10k price range, software RAID nowadays seems on par with hardware RAID. The programme SoftRAID has some monitoring tools for disk status. And in my case it's fairly simple: using RAID0 for speed only, I assume it will fail at some point, so backups should always be in place.

Btw, if you're interested - planning my software RAID0 like this now:

4-bay JBOD enclosure, USB 3.0 and/or eSATA
3 or 4 x Seagate Barracuda ST4000DM005 4TB

Why these drives? Because they have an average seq read speed of 155 MB/s (source: userbenchmark.com), and four of them makes 620 MB/s total seq read speed in RAID0. Which is also the max speed of USB 3.0.

That will be way enough for multiple editing streams of 1080p material, but more importantly should also be able to handle a couple of heavier streams / codecs in 4K material (Sony F65 material f.i). And 12 TB / 16 TB should be enough to store hours and hours of low-res ProRes proxies, keeping 50% free for optimal disk speed.

Sorry for the exposé, wanted to run my thinking past you now we're on the topic.

 

[casperes1996]

I'm curious to hear why you still feel hardware RAID might be better suited for my needs?

Just that they're more error-prone and if you insert them in another system the likelihood of them just working is really low. But with the following in mind, I don't think it's a problem for your workflow

backups should always be in place.

If the backups are there, I don't see any issue using software RAID - well, aside from one, but it's minor and probably not enough that you want to spend the cash for a hardware RAID system - Overhead. Performance will be a tad lower with software RAID because of the added overhead. The CPU will need to do a bit of extra work, and the drive speed will be a tad lower, but it's marginal.

I'm using Premiere Pro CC and a bit of Davinci Resolve 14.

Not a big fan of the pricing structure of CC, but I see it as good for some, especially those who make money off of the software. – I absolutely love Black Magic's stuff though, and for colour work especially, Resolve is lovely. Their recent updates have been really awesome as well, increasing performance significantly under circumstances like stabilisation and adding a nice couple of new features for the paid release, like a surprisingly good auto tool for retouching face details.

Anyway that was a slightly unrelated rant. Hehe

 

[ColdCase]

The OWC thunderbay enclosures are very good (4 bay). They used to be a good deal but seem to increased in price. For heavy duty workflows, the Areca stuff is great. They can be run in JBOD mode or a number of different RAID combos.

For sizing you probably want to have twice as much RAID storage as you need as they can slow down dramatically as they fill.

 

[hieronymuspool]

The OWC thunderbay enclosures are very good (4 bay). They used to be a good deal but seem to increased in price. For heavy duty workflows, the Areca stuff is great. They can be run in JBOD mode or a number of different RAID combos.

For sizing you probably want to have twice as much RAID storage as you need as they can slow down dramatically as they fill.

Thanks for the pointers! Will probably go with a cheap 4 bay JBOD enclosure for now - and regret it later.

Not a big fan of the pricing structure of CC, but I see it as good for some, especially those who make money off of the software. – I absolutely love Black Magic's stuff though, and for colour work especially, Resolve is lovely. Their recent updates have been really awesome as well, increasing performance significantly under circumstances like stabilisation and adding a nice couple of new features for the paid release, like a surprisingly good auto tool for retouching face details.

Anyway that was a slightly unrelated rant. Hehe

Agreed on all aspects! Unfortunately I also need to be able to exchange work with other editors and most of them usually still mainly work in PP as well. But whenever I have shorter solo projects I'll do them on Davinci. Might pay for the Studio version when I need to do a 4K timeline or some denoising, but let's cross that bridge when we get there.

 

[casperes1996]

need to do a 4K timeline

Pretty sure 4K projects were actually added to the free version in one of the recent updates!

 

[hieronymuspool]

Pretty sure 4K projects were actually added to the free version in one of the recent updates!

Woops, better update then!

 

[joema2]

...software RAID nowadays seems on par with hardware RAID....

This is correct, I edit large documentaries and have several 32TB OWC Thunderbay 4 arrays in RAID-5 or RAID-0 using SoftRAID. I also have a Promise Pegasus R4 and those are very good but the OWC Thunderbay units are equal or faster than similar hardware RAID configurations and (unlike hardware RAID) it doesn't lock you into a proprietary RAID format.

If a hardware RAID chassis fails you can generally only use the drives in another chassis of the same brand and type. The low-level disk format is unique to that proprietary RAID controller. By contrast, if a chassis fails when using SoftRAID or Apple RAID you can use those drives in any other manufacturer's chassis. With SoftRAID or Apple RAID, if you want to change brands you can -- you're not locked in.

For RAID-0 you actually don't need SoftRAID for this since macOS supports it but I prefer SoftRAID due to the error reporting and flexibility.

Since RAID-1, RAID-5 or other redundant formats are not backups, you must back up everything anyway. Due to this requirement I have gradually switched over to RAID-0 and generally use an identical backup array via Carbon Copy.

 

[hieronymuspool]

This is correct, I edit large documentaries and have several 32TB OWC Thunderbay 4 arrays in RAID-5 or RAID-0 using SoftRAID. I also have a Promise Pegasus R4 and those are very good but the OWC Thunderbay units are equal or faster than similar hardware RAID configurations and (unlike hardware RAID) it doesn't lock you into a proprietary RAID format.

If a hardware RAID chassis fails you can generally only use the drives in another chassis of the same brand and type. The low-level disk format is unique to that proprietary RAID controller. By contrast, if a chassis fails when using SoftRAID or Apple RAID you can use those drives in any other manufacturer's chassis. With SoftRAID or Apple RAID, if you want to change brands you can -- you're not locked in.

For RAID-0 you actually don't need SoftRAID for this since macOS supports it but I prefer SoftRAID due to the error reporting and flexibility.

Since RAID-1, RAID-5 or other redundant formats are not backups, you must back up everything anyway. Due to this requirement I have gradually switched over to RAID-0 and generally use an identical backup array via Carbon Copy.

Thanks for chiming in, joema2. Interesting to hear your experiences regarding the Promise R4 and OWC Thunderbays with SoftRAID as well.

Could I pick your brain regarding a Thunderbolt vs USB 3.0 JBOD enclosure in software RAID0, using 4 disks?

With 4 drives, running at around 155 MB/s average sequential read speed, you reach a theoretical max of 4 x 155 MB/s = 620 MB/s. USB 3.0 max theoretical speed is 5 Gbit/s / 8 = 625 MB/s. In other words: USB 3.0 would be ideal for this setup, Thunderbolt (1250 MB/s and up, depending on TB version) would be overkill.

Therefore I've been looking at cheap USB 3.0, 4-bay, JBOD enclosures - Startech / Mediasonic / Fantech / ICY BOX etc. I was about to pull the trigger, until I thought of real-world performance vs theoretical performance. Even though USB 3.0 should in theory be able to deliver a blazing 600 MB/S or-so read speed, is this really the case in real life, especially with cheap JBOD enclosures such as those? Isn't this where the native build quality and technology behind Thunderbolt would give you those speeds much quicker?

A RAID0, 4-bay, USB 3.0 vs Thunderbolt speed test can be found here (comparison chart here). A cheap Startech enclosure (perhaps a hardware RAID model, not software RAID) was used to test USB 3.0 speeds, which with 4 disks only reached around 250 MB/s (!) - way lower than the theoretical speed would suggest. The Promise R4 in Thunderbolt delivered around 500 MB/s.

In other words: even though the theoretical speed suggests it would be overkill for my needs, should I consider purchasing a Thunderbolt enclosure rather than a USB 3.0 enclosure?

Note: the above question is in the context of reading (not writing) large video files. My RAID0 setup will almost exclusively be read from by my NLE during video editing, which is why I'm calculating with average sequential read speeds of HDD's.

 

[ColdCase]

Its more than speed, its also quirky protocol stacks and reliability. My experience, is that loading up with of USB type disk devices will only lead to frustration. TB is PCI and disk friendly, so you skip all the protocol overhead. My USB enclosures are relegated to backup storage. My recent media and projects are on TB SSD or internal, less used on a fast RAID0 array which can try my patience after working with the recent media.

BTW, I use the RAID software built into disk utility. I only use RAID0 and mirror as with todays cheap storage, more sophisticated RAID is much more trouble than they are worth for my use case.

 

[joema2]

...a Thunderbolt vs USB 3.0 JBOD enclosure in software RAID0, using 4 disks?...

I agree with ColdCase. It's not about the speed. A four-drive Thunderbolt array will likely be more reliable than a bare-bones USB 3 array. You also must think about future expansion. Thunderbolt lets you daisy-chain arrays, whereas USB is point-to-point.

With video you always think "I'll never need more than *that* much capacity", but you keep requiring more. Expansion with Thunderbolt is easier. When you are talking multiple high-capacity drives, they dwarf the cost of the chassis.

Like ColdCase I have several USB arrays but I only use them for tertiary backup. I usually back up from one Thunderbolt array to another one.

However if you want to try a StarTech USB enclosure, go ahead. Maybe it will work fine. If it doesn't work you could pull the drives and use them in a Thunderbay 4. But I would personally fear a flaky intermittent problem due to protocol or driver or weak power supply in the JBOD chassis that might cost time and data.

 

[hieronymuspool]

Thanks for the great input, both. This is, as yet, a non time-critical purchase for me, so I have the time to really figure out a good approach for my needs.

As it stands, in the past days I've shifted from going with a cheap USB JBOD enclosure, to saving up a bit and investigating a 4-bay Thunderbolt JBOD or RAID enclosure. Arguments are indeed stability and future-proofing / scalability, rather than short-term cost benefits.

Though if I suddenly get eager I indeed might still try out a relatively cheap USB 3.0, UASP-enabled, JBOD enclosure and see where it might lead.

 

[BeechFlyer]

As it stands, in the past days I've shifted from going with a cheap USB JBOD enclosure, to saving up a bit and investigating a 4-bay Thunderbolt JBOD or RAID enclosure. Arguments are indeed stability and future-proofing / scalability, rather than short-term cost benefits.

Facing that same decision a few months ago, I bought the Drobo 5D3 for my new iMac. I couldn't be happier with it.

 

[Mac_User 0101]

I'm hoping to revive this thread. I'm in the market for a 4 bay, USB 3.x rack mountable RAID enclose. Onboard controller supporting RAID 5 & 10 is a plus but don't mind using SoftRaid. There aren't a lot options fitting that criteria so I'm about to buy the OWC Mercury Rack Pro but I'm feeling guilty spending the money on that and not SSD's.

I'm using a classic Mac Pro 5,1 with 10Gb USB card and all ready have a Squid with 4 m.2 internal. It's not enough capacity and I'd like redundancy as well. Performance is not so much the issue. I'd take an enclosure fitting that criteria that houses 4 or more 2.5" bays and throw a bunch of SATA SSD's in there being that they are really affordable now and available in larger capacities. A lot of what I'm finding is SAS or Thunderbolt. Any suggestions?

 

[ColdCase]

SoftRaid with USB drives is not very satisfying in heavy duty use environments like video editing. Enclosures with built in HW RAIDs are less of a headache. Pick SSDs that are RAID tolerant/compatible.

 

[Mac_User 0101]

SoftRaid with USB drives is not very satisfying in heavy duty use environments like video editing. Enclosures with built in HW RAIDs are less of a headache. Pick SSDs that are RAID tolerant/compatible.

I should have explained in my first post that it's not for video editing. It was hard to find a thread that even remotely matched what I'm looking to do. I'm an audio engineer and I do work off of multiple ssd's but the RAID I'm seeking is for mass storage backup. At least 8 TB of performance RAID and another 8TB redundancy. I'd like it to be external because of the SATA 2 limitation of my Mac Pro and all my PCIe slots are filled. I do have USB 3.1 gen 2 with dedicated 10Gbps controllers and eSATA. The enclose should fit in my rack and be quiet because it will be in my control room. I was actually thinking of picking up the OWC Mercury rack pro which doesn't use trays and fitting the ssd's with something like the startech 2.5" to 3.5" adapter. What do you think about that or do you have a better enclosure in mind to suite my needs?

 

[ColdCase]

I abandoned USB for main storage years ago, just more trouble than its worth. I've been using OWC, LaCie, and Oyen Thunderbolt Enclosures mostly in JBOD or mirror mode. They just work. So I dunno where the current market is on USB enclosures.

I've used the 3.5 to 2.5 adapters and they work just fine. Its just SATA, after all. Some SSDs include an adapter plate in their packaging that will sometimes work.

 

[iluvmacs99]

I'm hoping to revive this thread. I'm in the market for a 4 bay, USB 3.x rack mountable RAID enclose. Onboard controller supporting RAID 5 & 10 is a plus but don't mind using SoftRaid. There aren't a lot options fitting that criteria so I'm about to buy the OWC Mercury Rack Pro but I'm feeling guilty spending the money on that and not SSD's.

I'm using a classic Mac Pro 5,1 with 10Gb USB card and all ready have a Squid with 4 m.2 internal. It's not enough capacity and I'd like redundancy as well. Performance is not so much the issue. I'd take an enclosure fitting that criteria that houses 4 or more 2.5" bays and throw a bunch of SATA SSD's in there being that they are really affordable now and available in larger capacities. A lot of what I'm finding is SAS or Thunderbolt. Any suggestions?

You can try this..

https://www.vantecusa.com/products_detail.php?p_id=33&p_name=NexStar+HX4R&pc_id=2&pc_name=3.5"+Enclosures&pt_id=1&pt_name=Hard+Drive+Enclosures

It is a 3.5" 4 disk Raid enclosure USB 3.0 or eSATA supporting RAID 0,1,5 and 10 for not a lot of money. You can buy a 2.5" to 3.5" disk converter if you plan to use SSDs. Plus it is OSX compatible. The RAID rebuilt time for unit like this is around 1Gb/min or 1Tb/24 hrs (to be conservative if you're accessing data). I don't have this unit; rather I own the more expensive Drobo 4 drive RAID and a Netgear 4 drive RAID NAS Gigabit and pair them together for dual and triple redundancy (meaning copying files from Drobo 4 to Netgear via Carbon Copy Cloner for backup). I used it mainly for video and audio work in the past when I was working in the digital media industry.

Don't forget that the RAID unit can fail as well as the disks in it, so you want 2 different RAID units or at least made at different times so both don't fail at the same time.

The reason I would recommend the hardware option is with RAID failure. When you have a disk failure, the hardware RAID will go on to rebuild your drive array in the background while not affecting the Mac Pro at all. In fact, you can turn off the Mac Pro while letting the hardware RAID box rebuild the array saving you time and electricity. With 8TB with a 4TB dual redundancy, you can expect around 4 to 5 days of full rebuild or sooner (for a single 4TB drive), so you can see the appeal of the standalone hardware RAID. And that's why I went with hardware. I started with Softraid in the past.

Personally, I love my RAID setup as I had a RAID failure twice over a span of 8 years and RAID had saved my rear on my audio and video files which for a single drive would be a devastating loss. I went with RAID because of the one drive failure on my storage drive that held my precious travel memories from a number years back that I didn't have a backup for. Though the only backup I had of those photos and videos are in my own memory.

The Vantec option is a much cheaper option than Drobo or Netgear, but the manual is atrocious. You can spend more of course, but it gets uber expensive with higher end options. I got the uber setup thanks to my former company paying for the gear and allowing me to keep them after leaving them.

Hope this helps..

 

[joema2]

...Don't forget that the RAID unit can fail as well as the disks in it, so you want 2 different RAID units or at least made at different times so both don't fail at the same time...

This is excellent advice. The data on a redundant RAID can be lost due to chassis failure, OS filesystem error, user error, etc.

...The reason I would recommend the hardware option is with RAID failure. When you have a disk failure, the hardware RAID will go on to rebuild your drive array in the background while not affecting the Mac Pro at all....

RAID rebuild can in some cases take a very long time. I ran many rebuild tests on my former 8TB Pegasus R4 RAID-5 array. Rebuild time was very sensitive to strip size, and at 128k strip size it was extremely slow: https://photos.smugmug.com/photos/i-pdz6p2j/0/632a78ee/O/i-pdz6p2j.jpg

I tested SoftRAID rebuild time and don't have the figures but it was much faster. On the hardware RAID system, performance during the rebuild phase was so slow it was not practically usable. I don't remember SoftRAID performance during rebuild but it didn't take as long.

Since (as you said) you really need a duplicate array I have since quit using RAID-5 and just use RAID-0 on two arrays which are synchronized nightly with Carbon Copy or other tools. That way if a failure happens I can be back at full performance within minutes vs hours or days of degraded performance during the rebuild.

For video editing this works well since they don't change that often. The editing metadata in the library is on a separate drive that is continuously backed up via Time Machine.

If you had a large RDBMS that approach wouldn't work since the database is constantly changing. So what backup system and method works for one workflow may not work for another.