Thunderbolt 2K Capture vs Recorder

[olindacat]

http://www.blackmagic-design.com/products/ultrastudio3d/

So how does this thing work? Does it connect to the MacBook Pro? Are SSDs needed to keep up with the stream? Does this record or just pipe to a program? I don't see much on the site....

Am weighing this vs. a portable recorder, e.g., Ki Pro Mini or Nano

 

[LethalWolfe]

http://www.blackmagic-design.com/products/ultrastudio3d/

So how does this thing work? Does it connect to the MacBook Pro? Are SSDs needed to keep up with the stream? Does this record or just pipe to a program? I don't see much on the site....

Am weighing this vs. a portable recorder, e.g., Ki Pro Mini or Nano

It provides an external interface to capture from SDI and HDMI sources. It connects to the host computer via ThunberBolt and your storage needs will be dictated by your capture settings in whatever NLE you are using. If you capture as ProRes4444 you'll need beefier storage than if you capture as DVCPro HD. The device is just a pipe, not a recorder, so if you are looking for a box to strap to your camera and go shooting this is not it.

Something from Blackmagic that would be comparable to the KiPro Mini is the HyperDeck Shuttle.


Lethal

 

[olindacat]

It provides an external interface to capture from SDI and HDMI sources. It connects to the host computer via ThunberBolt and your storage needs will be dictated by your capture settings in whatever NLE you are using. If you capture as ProRes4444 you'll need beefier storage than if you capture as DVCPro HD. The device is just a pipe, not a recorder, so if you are looking for a box to strap to your camera and go shooting this is not it.

Something from Blackmagic that would be comparable to the KiPro Mini is the HyperDeck Shuttle.


Lethal

I thought so. I know about the Shuttle and it is very affordable. (Actually ordered the Ki Pro Mini but am bummed about the single card/power gooble/lack of monitor.... Wondering if you could use this with CS5 OnLocation and Premiere and a MacBook Pro to BE recorder/monitor.

 

[martinX]

A question that might be better addressed to their sales department.

 

[KeithPratt]

I've read your other thread and I think you're going a bit overboard.

There are a lot of products in videography that are marketed as workflow streamliners; but crowbarring more and more of them into your workflow has the opposite effect.

The FS100 records AVCHD internally, and if you're going for a natural look that will probably yield an acceptable image. ProRes is the next step up, but considering uncompressed seems pretty ridiculous to me. If you want a recorder with a screen, take a look at the Sound Devices PIX 220.

Whilst I'm here I'll give you my thoughts on your other thread. I think you've made that overly complicated too, which is particularly disastrous for a back-up strategy. What I'd recommend is:

OS Drive
OS Clone (single-disk FW800 external)
Scratch (FW800 hardware RAID5)
Working Back-up (single-disk FW800 external/iMac second internal)
Archive A (single-disk FW800 external)
Archive B (single-disk FW800)

The OS Clone and the RAID parity of Scratch minimise potential downtime and frustration; but don't consider them back-up. Instead, as soon as you get back from a shoot, copy your video to Working Back-up, and as you edit save your project files and any original media you create during the process as you go.

Once you've finished a project, copy project files, original media and master to Archive A and B, and delete from OS Drive, Scratch and Working Back-up.

Using two separate drives for archive means you can keep them in two different locations, and copying files separately to each from the source leaves you less at risk from Silent Data Corruption and bit rot than RAID1 would.

You can substitute FW800 for Thunderbolt as devices become available.

 

[olindacat]

I've read your other thread and I think you're going a bit overboard.

There are a lot of products in videography that are marketed as workflow streamliners; but crowbarring more and more of them into your workflow has the opposite effect.

The FS100 records AVCHD internally, and if you're going for a natural look that will probably yield an acceptable image. ProRes is the next step up, but considering uncompressed seems pretty ridiculous to me. If you want a recorder with a screen, take a look at the Sound Devices PIX 220.

Whilst I'm here I'll give you my thoughts on your other thread. I think you've made that overly complicated too, which is particularly disastrous for a back-up strategy. What I'd recommend is:

OS Drive
OS Clone (single-disk FW800 external)
Scratch (FW800 hardware RAID5)
Working Back-up (single-disk FW800 external/iMac second internal)
Archive A (single-disk FW800 external)
Archive B (single-disk FW800)

The OS Clone and the RAID parity of Scratch minimise potential downtime and frustration; but don't consider them back-up. Instead, as soon as you get back from a shoot, copy your video to Working Back-up, and as you edit save your project files and any original media you create during the process as you go.

Once you've finished a project, copy project files, original media and master to Archive A and B, and delete from OS Drive, Scratch and Working Back-up.

Using two separate drives for archive means you can keep them in two different locations, and copying files separately to each from the source leaves you less at risk from Silent Data Corruption and bit rot than RAID1 would.

You can substitute FW800 for Thunderbolt as devices become available.

Thanks, Keith. I'm sure you are right, about over-thinking. But when I don't know, I tend to do that. Your single disk BU solution is very helpful, only I am currently faced with many one-disk FW drives on my desk because I cannot fit what I have on all of them. They are fickle about mounting and often die. Because I need to access even work I did 4-5 years ago and have about 4TB of archives and am now shooting HD, the Qx2's 9TB of Raid 5 storage needs seem to be much more in line with what I need. Now, if what I need and what I should get are what is being debated, then I defer to you, but still need more than what two single 3TB drives can offer.

Regarding the TB capture: I have been a working photographer for almost three decades and have been taping a golf show here for the past three years. I have used the most basic gear, camera, and tape work flow I can to learn all of its limitations with my NLE being Premiere, and my deliverable being everything from the web, to broadcast. I can tell you that it isn't the content. I can resell the footage I am shooting if I can meet some agency super-high expectations.

Also, I don't want to limit myself. I know I could use the Sony Codec in most cases for my show, as well as a proxy for agencies if I have a 2K master, but confining myself is where I don't want to go. So, I asked here, what others thought.

Thank you for reading my other posts and for consolidating your replies on this one. I didn't now about the PIX recorder. It is very nice. I don't see a price on their website, and 'Summer' is the delivery date. I also can't shoot 1080p60 with it (I don't think) which is what the FS100 can do. HD workflow is entirely new to me. Forgive my ignorance or multiple posts.

 

[olindacat]

A question that might be better addressed to their sales department.

Went to their website and will call. I just ask here because what people experience in real life vs. what a salesperson will tell you can often be two totally different things, martinx.

 

[martinX]

Went to their website and will call. I just ask here because what people experience in real life vs. what a salesperson will tell you can often be two totally different things, martinx.

From what I am led to believe, BlackMagic is a professional company interested in creating a relationship with a customer, rather than making an individual sale.

Also, posting on Creative Cow may yield more users of BM products than here.

 

[martinX]

I didn't now about the PIX recorder. It is very nice. I don't see a price on their website, and 'Summer' is the delivery date. I also can't shoot 1080p60 with it (I don't think) which is what the FS100 can do. HD workflow is entirely new to me. Forgive my ignorance or multiple posts.

I didn't know about it either, yet someting like the Atomos Ninja has been getting press for 6 months before it came out. Funny how some things slip under the radar and others can't be missed.

Anyway the Sound Devices Portable Video Recorder PIX 220 seems to be going for $1600, not available yet.

 

[LethalWolfe]

Considering that ThunberBolt peripherals are brand new (are any of them even shipping yet) your best bet is to contact the companies directly (in this case both BM and Adobe) and see what it's going to take for the two products to play nice.


Lethal

 

[olindacat]

Considering that ThunberBolt peripherals are brand new (are any of them even shipping yet) your best bet is to contact the companies directly (in this case both BM and Adobe) and see what it's going to take for the two products to play nice.


Lethal

I'll do that, of course. Creative Cow is super martinx—I am there a lot. But I have to say inasmuch as other forums are rich resources, this one is among the best and I would put nothing past the people here, in terms in knowledge..... I never posted here but have been reading this forum for years.

 

[olindacat]

Keith: this is very helpful and I just want to say thanks for your time and suggestions....

The OS Clone and the RAID parity of Scratch minimise potential downtime and frustration; but don't consider them back-up.

So are you saying to NOT buy the Qx2? Or, buy it and RAID 5 just for Scratch? That's 9TB of scratch? (I'm sorry, I'm not understanding you.)

Instead, as soon as you get back from a shoot, copy your video to Working Back-up, and as you edit save your project files and any original media you create during the process as you go.

Why wouldn't I want to work on project on SDD? Why would I want to render from video stored on Working Back-up FW drive vs having project files on OS SSD? Seems like I'm not leveraging the speed of the SSD if all my current work/video files are on an external Working Back-up FW drive. NO?

Once you've finished a project, copy project files, original media and master to Archive A and B, and delete from OS Drive, Scratch and Working Back-up.

Is Working Back-up the RAID 5 then? Sorry, I'm very brain-frozen most of the time and this is just taking a little to sink in...

Using two separate drives for archive means you can keep them in two different locations, and copying files separately to each from the source leaves you less at risk from Silent Data Corruption and bit rot than RAID1 would.

You can substitute FW800 for Thunderbolt as devices become available.

This is very helpful (again). I just need more space than one drive can handle. I suppose I can buy 4x3TB portables? Many thanks, Keith! (So much for thinking I had it all sussed out.....)

 

[martinX]

I'll do that, of course. Creative Cow is super martinx—I am there a lot. But I have to say inasmuch as other forums are rich resources, this one is among the best and I would put nothing past the people here, in terms in knowledge..... I never posted here but have been reading this forum for years.

The Cow beats us for sheer size, though, and with that size comes a depth of expertise that assures you that someone, somewhere on that sprawling ranch has had the same question as you.

 

[KeithPratt]

So are you saying to NOT buy the Qx2? Or, buy it and RAID 5 just for Scratch? That's 9TB of scratch? (I'm sorry, I'm not understanding you.)

I'm saying get a RAID5 Scratch, but don't consider it back-up. That's what the Working Back-up is for. (You're not working from the Working Back-up.)

I suppose I can buy 4x3TB portables?

That's what I was originally suggesting — when you fill up one pair, buy another. For most it'd be a waste of resources to have an online RAID5 archive, but if you're going back to previous work really often it might be worth it.

I also can't shoot 1080p60 with it (I don't think) which is what the FS100 can do.

Probably not. But I don't think there are many external recorders that can.

 

[olindacat]

I'm saying get a RAID5 Scratch, but don't consider it back-up. That's what the Working Back-up is for. (You're not working from the Working Back-up.)

IQ deficit here Keith. Why do I wan to waste 9TB of drive space on scratch? I thought scratch was virtual memory. I'll have 16GB of ram. Why are you telling me to do scratch raid 5? I Googled it but found very little so I'm sorry to be asking 3X.

That's what I was originally suggesting — when you fill up one pair, buy another. For most it'd be a waste of resources to have an online RAID5 archive, but if you're going back to previous work really often it might be worth it.

80% of what I do is regurgitate and reuse archives. That was why I was wanting to have an archive that was very accessible. But now you are telling me to make the entire raid 5 a scratch disk, or at least that's sure how it sounds, and I'm lost as to your reasoning.

 

[olindacat]

I called BM today asking how I could make use of their new UltraStudio 3D. I understand it is very new tech, but he was telling me to search the net for my own solutions. When I questioned him as to why he couldn't give me a more substantive response, he huffed and told me to look at external HDD solutions, e.g., CalDigit. I was pretty surprised as I was hoping that BM would be fielding questions from morons like me on a regular basis. I'll go talk to the cows now.

 

[KeithPratt]

Why do I wan to waste 9TB of drive space on scratch? I thought scratch was virtual memory. I'll have 16GB of ram.

In video editing, a scratch disk is where you house all of your project's video and audio.

80% of what I do is regurgitate and reuse archives. That was why I was wanting to have an archive that was very accessible.

Then I'd suggest getting a big scratch and keeping all the video you're going to be regularly going back to on it. A RAID5 is not infallible though, and you shouldn't treat it as a back-up, so archive your projects to single-disk FW800 drives too.

I called BM today asking how I could make use of their new UltraStudio 3D. I understand it is very new tech, but he was telling me to search the net for my own solutions. When I questioned him as to why he couldn't give me a more substantive response, he huffed and told me to look at external HDD solutions, e.g., CalDigit. I was pretty surprised as I was hoping that BM would be fielding questions from morons like me on a regular basis.

Its basic function is to capture and play out video. This particular model's key features are that it can do 3D and uses the Thunderbolt interface (meaning you can use it with MacBook Pros and iMacs, which wouldn't otherwise be possible). But you could use it in a thousand slightly different ways, so I can understand him not knowing how to answer such an open-ended question.

 

[olindacat]

In video editing, a scratch disk is where you house all of your project's video and audio.

I have been editing for the past three years (past one much more) never knowing this. I never moved any files onto any scratch disk or into any scratch folder. I just spent $1200 to buy the Qx2 thinking it'd be a BU/archive. According to what I am understanding, this is now going to be 100% scratch, is that correct? Does this mean I should NOT buy the two 240GB Vector 3 SSDs and do RAID 0 stripe for my OS and 'working' projects? I thought that would be the way to go vs. the slower FW800. Still don't get how a $1200 9TB FW800 RAID5 scratch is better.

Then I'd suggest getting a big scratch and keeping all the video you're going to be regularly going back to on it. A RAID5 is not infallible though, and you shouldn't treat it as a back-up, so archive your projects to single-disk FW800 drives too.

I was essentially going to do that but not as scratch, but as a working archive that I'd BU to externals. Isn't scratch temporary? Does the computer or Premiere dump it whenever it wants? I still don't get the scratch concept and as looking in Google to no real avail.

Its basic function is to capture and play out video. This particular model's key features are that it can do 3D and uses the Thunderbolt interface (meaning you can use it with MacBook Pros and iMacs, which wouldn't otherwise be possible). But you could use it in a thousand slightly different ways, so I can understand him not knowing how to answer such an open-ended question.

I asked him if I could capture to a new MacBook Pro and he said the HDD/SSD wasn't fast enough for the incoming video stream, and that I'd need a RAID 0 external. That much I got, but thought the conversation would have gone a little better given that I am in the market for an optimum recording/capture solution, even if I don't know what I'm doing about scratch, etc.

Thank you Keith.

 

[KeithPratt]

Does this mean I should NOT buy the two 240GB Vector 3 SSDs and do RAID 0 stripe for my OS and 'working' projects? I thought that would be the way to go vs. the slower FW800. Still don't get how a $1200 9TB FW800 RAID5 scratch is better.

Your Mac will boot a little quicker and FCP will open a little speedier with SSDs, but they're not worth their price for your purposes. All your scratch is doing is streaming video to and from disk. The Qx2 RAID will get you three concurrent streams of ProRes over FW800. If that's enough for you, there's little to be gained from faster disks.

Have a read of the FCP online manual.

Isn't scratch temporary? Does the computer or Premiere dump it whenever it wants?

No, you decide when to delete things.

EDIT: Just remembered you're using Premiere (for now at least). Principles still apply, and there's plenty of general information to glean from the FCP manual.

 

[wonderspark]

Don't get hung up on terms. In my opinion, "scratch" is just a description for where I put renders, temp files, and any data that has to be read/written very fast, then deleted when I'm done with it. "Working" disks/folders are where I keep files I'm currently using for a project. Scratch and working can be the same disk, but I tend to keep the temp stuff separate from the working stuff by using different folders on the disk.

I think you'll be fine with the 9TB RAID5 drive as your source for all the files you keep reusing on projects, and 2x 240GB SSDs in RAID0 as your working disks, if 480GB is enough space for you to work with. (It's not enough for my project right now, where I'm editing three months of time-lapse into a single HD timeline. The video resolution is about the same as editing RED... at 4928 × 3264, so my working files are 3.35TB!) If 480GB is NOT enough space to edit with, I'd save some money and just put 3x 2TB drives into the Mac Pro in RAID0 for a working disk of 6TB, or RAID5 as 3.92TB (which is what I have) and use the externals for backups.

OSX and programs like Adobe can be on a normal, single HDD, apart from your other data. An SSD there will only boot faster, and be of little benefit otherwise.

Also, don't forget to keep one external drive for the OSX / program drive clone.

 

[wonderspark]

I forgot you have the iMac instead of the Mac Pro.
In that case, you should set up a working drive with Thunderbolt somehow, like with this thing.

 

[olindacat]

I forgot you have the iMac instead of the Mac Pro.
In that case, you should set up a working drive with Thunderbolt somehow, like with this thing.

Hi Ed Great you popped in. I was all set to move ahead with the 240x240 0 stripe until I got my head handed to me. Scratch was, I thought, a temp file. That was what I remember from the old days. I know where it is and that is about it. Now, perhaps video handles scratch differently. I never look at the scratch folder nor did I ever think I needed to.

Say you are doing a scene in your movie. You have the scary girl down the dark hall. You've applied effects and for argument's sake you've got too many tracks on your timeline so you decide to do an intermediate render, save that first timeline if you need to go back to it, and then make a new (clean timeline) w/intermediate render of girl/effects so you can add even more effects/tracks.

Is the scratch space where the processing of that render is happening? Does it spool (to use an old term) there? (I understand that concept.)

I did read the FCP manual on disk space, but it doesn't calculate scratch space, but file space based on bit rates. So, I still don't know how much scratch I'd need. But, I can say for the :30 minute show, we're talking about far less than my :90 minute movie. The difference between each will be the graphics. The movie will have very few graphics, comparatively.

Do you just store your intermediate render in a project sub-folder in such a situation? Import that into the new timeline? Or, is there a copy of it in this scratch folder?

 

[olindacat]

I think you'll be fine with the 9TB RAID5 drive as your source for all the files you keep reusing on projects, and 2x 240GB SSDs in RAID0 as your working disks, if 480GB is enough space for you to work with.

So here's where I get a little into gray areas:

Volume One (Raid 0)
SATAIII SSD1
SATAIII (cable from HDD used for 2nd SSD on Opt Bay) SSD2

Volume Two (or Two & Three)
SATAII Apple 2TB HDD (cable from Opt Bay)

Now, do I partition the 2TB HDD and put the OS in one, and the data on another? One person suggested I made the HDD a scratch disk.

Or, can I just have the OS on Volume I and project files too. As they are only there one month at a clip, and backed up daily....

I was going to use Volume Two as data for all else but video.

(It's not enough for my project right now, where I'm editing three months of time-lapse into a single HD timeline. The video resolution is about the same as editing RED... at 4928 × 3264, so my working files are 3.35TB!)

You gotta show me a clip of that!

OSX and programs like Adobe can be on a normal, single HDD, apart from your other data. An SSD there will only boot faster, and be of little benefit otherwise.

I know. But I'm trying to figure out where to put OS given this scenario. (I know once TB devices come online that's change all of this.)

Also, don't forget to keep one external drive for the OSX / program drive clone.

Will do. Will also buy MP when time comes. Like those Promise Raids been sniffing at them for months

 

[wonderspark]

Hi Ed Great you popped in. I was all set to move ahead with the 240x240 0 stripe until I got my head handed to me. Scratch was, I thought, a temp file. That was what I remember from the old days. I know where it is and that is about it. Now, perhaps video handles scratch differently. I never look at the scratch folder nor did I ever think I needed to.

Say you are doing a scene in your movie. You have the scary girl down the dark hall. You've applied effects and for argument's sake you've got too many tracks on your timeline so you decide to do an intermediate render, save that first timeline if you need to go back to it, and then make a new (clean timeline) w/intermediate render of girl/effects so you can add even more effects/tracks.

Is the scratch space where the processing of that render is happening? Does it spool (to use an old term) there? (I understand that concept.)

I would put that intermediate file into a dedicated folder for intermediate files on my working drive, since the intermediate files would become new working files. Otherwise, I would render the effect-filled timeline so it plays real-time, and tell Adobe Premiere to use the scratch drive for the render files. You can use the same fast drive for "working' and "scratch" files, but keeping them apart will help with organization.

I did read the FCP manual on disk space, but it doesn't calculate scratch space, but file space based on bit rates. So, I still don't know how much scratch I'd need. But, I can say for the :30 minute show, we're talking about far less than my :90 minute movie. The difference between each will be the graphics. The movie will have very few graphics, comparatively.

Do you just store your intermediate render in a project sub-folder in such a situation? Import that into the new timeline? Or, is there a copy of it in this scratch folder?

I keep timeline renders and working/intermediate files in their own folders so I don't have to wade through a mess to find something.

So here's where I get a little into gray areas:

Volume One (Raid 0)
SATAIII SSD1
SATAIII (cable from HDD used for 2nd SSD on Opt Bay) SSD2

Volume Two (or Two & Three)
SATAII Apple 2TB HDD (cable from Opt Bay)

Now, do I partition the 2TB HDD and put the OS in one, and the data on another? One person suggested I made the HDD a scratch disk.

Or, can I just have the OS on Volume I and project files too. As they are only there one month at a clip, and backed up daily....

I was going to use Volume Two as data for all else but video.

I'll qualify this by saying you're building a system I don't have direct experience with, but on the Mac Pro forum, there are some guys there like Nanofrog who will know more about how your disk cabling will work. So, provided the data throughput is sufficient as you've described the configuration:

I'd put the OS and program files onto the 2TB HDD. I have all kinds of stuff on that drive in my system, such as tons of sound effects files, music scores, images for graphics and lower thirds, etc. These files work fine here, and by using the 2TB drive for them, you won't use precious space on the much smaller SSD RAID you have going.

I'd use the 480GB SSD RAID0 for working files and scratch, in one partition and two main folders. When you find drive space is getting critical, nuke the scratch folder and re-render the timeline if you have to.

If 480GB isn't enough to work with, you're in a rough spot. Maybe the scratch could be moved to the 2TB drive your OS and programs are on in a pinch, while you finish the :30 minute program.

I guess this is why most of us tend to use Mac Pros.

 

[olindacat]

I guess this is why most of us tend to use Mac Pros.

I never doubted the wisdom of the Mac Pro. Recall that print is my main biz so I plan to repurpose the iMac and use as Bay 2 for some video and mostly print & photo editing, and then buy the MP when it has 'all that jazz' this fall.