External Drive for 4k realtime editing?

[pmj13]

Hey all,

I am beginning to work with a lot of very high-res files (4k/5k) and wanted some advice as to what the best external drive that could handle real-time 4k/5k editing would be.

I currently have a G-Safe (which is now discontinued, apparently) and am running it through FireWire 800. I know that this is nowhere near efficient enough for the file sizes I am working with.

Also, since the G-Safe isn't around anymore, is there a external drive system similar to it that does automatic backups of everything it writes?

Thanks!

Paul

 

[rick3000]

For 'automatic backups' I think you are referring to a RAID 1 mirror, where everything on one drive is duplicated on the second drive. There are externals that have two drives and do this, and many other backup solutions that can be debated endlessly. Drobo is one.

You don't say if you have Thunderbolt, obviously that is as fast as you can get with an external drive, but they are very expensive. The bottleneck is the connection type and you budget, obvious SSD via Thunderbolt would solve all problems. Or you could go with a RAID 0, two drives that act as one that will allow you to read from both once, effectively halving the read time for video but the connection could be a bottleneck.

What is your budget? Do you have TB?

 

[pmj13]

For 'automatic backups' I think you are referring to a RAID 1 mirror, where everything on one drive is duplicated on the second drive. There are externals that have two drives and do this, and many other backup solutions that can be debated endlessly. Drobo is one.

You don't say if you have Thunderbolt, obviously that is as fast as you can get with an external drive, but they are very expensive. The bottleneck is the connection type and you budget, obvious SSD via Thunderbolt would solve all problems. Or you could go with a RAID 0, two drives that act as one that will allow you to read from both once, effectively halving the read time for video but the connection could be a bottleneck.

What is your budget? Do you have TB?

Thanks for the response Rick. Other than my G-Safe, I have a 6TB G-RAID from 2011 with 0 Raid.

As for TB, i'm waiting for the new Mac Pro to come out next year, and then i'll have it. Right now, I have a 2009 MBP that's loaded up with all the options.

Once the 2013 Mac Pro is released, I plan on maxing that puppy out with as many cores, GB of RAM, etc as possible. I've been saving up, so until then, no; I don't have TB.

 

[rick3000]

Until you get TB, FW800 will be the bottleneck, the best way to max out the FW is to setup a RAID that stripes, or use SSD. Unfortunately, I think you will have have trouble at that resolution.

Are you shooting RED? Can't you use a workflow that edits the lower res files, then reconnect everything to the high res to export?

 

[pmj13]

Until you get TB, FW800 will be the bottleneck, the best way to max out the FW is to setup a RAID that stripes, or use SSD. Unfortunately, I think you will have have trouble at that resolution.

Are you shooting RED? Can't you use a workflow that edits the lower res files, then reconnect everything to the high res to export?

Great, thanks for the info.

Yes, I am using RED and arriRAW. That is what I was planning on doing, but the hard drive says it doesn't support ProRes 444...although I've heard of other people using prores 444 through the drive.

My 6tb is much quicker than the g-safe, though. Is there a way to daisy-chain the drives together, so that I can save something on my g-raid and it automatically backs itself up twice on my g-safe? That would help me a bit, I believe, as I am mainly using the g-safe for the reason that it backs everything up.

 

[rick3000]

You couldn't do a hardware solution as far as I know without repartitioning (and erasing the drives), there might be a software solution, or you can just copy the capture scratch and backup the editing files manually.

 

[mBox]

ATTO R680 SAS card with G-techs eS Pro 12TB.
We use that on Mac Pros for all our R3D files.
As fast as it is, you still need a ROCKET card to (we have two) playback in real time.

----------

...support ProRes 444...although I've heard of other people using prores 444 through the drive...

Its ProRes 4444.
We use that for R3D to ProRes conversion using REDCineXPro.
Thats for FCPX use but for Avid you can export to AMA from REDCineXPro.

----------

Are you shooting RED? Can't you use a workflow that edits the lower res files, then reconnect everything to the high res to export?

Yep this is common practice.
You can do low res ProRes for FCP legacy or higher res for FCPX.
For Avid you can export to DNxHD (any type depending on what codec you install) so you can use AMA.

 

[refison]

Sadly your FW800 port on your 2009 MBP is the limiting factor here. There is no solution that can beat your FW800 bottleneck save for removing your optical drive and installing two SSDs in its place in RAID 0, performance will be considerably noticable but it's still hampered at the 1.5gb/s bandwidth of SATA1.

The cost will be high, the storage size will be minimal, and the performance will still not be enough to edit raw 4/5k. Why not edit with the proxies like how people used to do with 10 bit uncompressed 1080i and online it after completion of edit?

Stick with your G-RAID until Mac Pro with thunderbolt and live with offline editing for a few months. The money you save now will be of better use on faster storage solutions in 2013. Btw, GTech and Lacie aren't on the same level in terms of speed when compared to other kids on the block such as CalDigit or Dulce to name a few within similar price bracket (a difference of $100-$200 for eSata/FW products)

 

[pmj13]

ATTO R680 SAS card with G-techs eS Pro 12TB.
We use that on Mac Pros for all our R3D files.
As fast as it is, you still need a ROCKET card to (we have two) playback in real time.

----------


Its ProRes 4444.
We use that for R3D to ProRes conversion using REDCineXPro.
Thats for FCPX use but for Avid you can export to AMA from REDCineXPro.

----------


Yep this is common practice.
You can do low res ProRes for FCP legacy or higher res for FCPX.
For Avid you can export to DNxHD (any type depending on what codec you install) so you can use AMA.

Great info, thank you so much.

Would this be the same practice for using arriRAW from the Alexa? (if you've had any experience with that format)

----------

Sadly your FW800 port on your 2009 MBP is the limiting factor here. There is no solution that can beat your FW800 bottleneck save for removing your optical drive and installing two SSDs in its place in RAID 0, performance will be considerably noticable but it's still hampered at the 1.5gb/s bandwidth of SATA1.

The cost will be high, the storage size will be minimal, and the performance will still not be enough to edit raw 4/5k. Why not edit with the proxies like how people used to do with 10 bit uncompressed 1080i and online it after completion of edit?

Stick with your G-RAID until Mac Pro with thunderbolt and live with offline editing for a few months. The money you save now will be of better use on faster storage solutions in 2013. Btw, GTech and Lacie aren't on the same level in terms of speed when compared to other kids on the block such as CalDigit or Dulce to name a few within similar price bracket (a difference of $100-$200 for eSata/FW products)

Thank you for the info! Yeah, you make some great points that I will most likely abide by. I just wish they'd hurry up and release the computer already! My MBP is having some serious trouble! haha

 

[mBox]

Great info, thank you so much.

Would this be the same practice for using arriRAW from the Alexa? (if you've had any experience with that format)

Not really just the ARRI files that Black Magic offered up for Resolve training.
They were ProRes files I do believe.
I work mostly with R3D, ProRes, MXF and soon CineRAW from the Black Magic Cinema Camera

 

[Hillskill]

Your at the end of the game where you need the fastest storage you can get. The larger the arrays the better the performance should be in theory. A year ago I installed a 120TB fibre channel XSAN which 10 Mac Pro FCPs shared simultaneous connection with.
The thing about the types of files you are editing with is that they are pretty heavy going on your system too, irrespective of bandwidth between your system and your storage. Most workstations that are working with R3D files tend to have a Red Rocket card for hardware accelaration when working with the files.

 

[wildgiles]

Does your MBP have an express slot? if so you can use the sonnet *pro* esata express card to at least double the connection speed to your hard drives through esata connection. With 4k/5k you really need to able to read/write at least 500megabytes/sec to your drives.

The only way to easily get the storage + speed you need through a MBP is to buy a MBP that has a least one thunderbolt connection.

 

[Hillskill]

My MBP has an express card slot with an esata adaptor. When it works it's great but it will often cause system lookups. Yet to try it on ML though.

 

[carestudio]

Hey all,

Also, since the G-Safe isn't around anymore, is there a external drive system similar to it that does automatic backups of everything it writes?

Thanks!

Paul

I have more than 15 units of CalDigit VR2 in my studio. Part of them are set to RAID 1 which is automatic backup method. I really love to use CalDigit's VR since they have drive modules and you can set the RAID levels as you want. Same chassis can do RAID 0, RAID 1 and JBOD. It also support Apple's latest USB 3 connection, speed is FAST!