Scratch Disk Selection - Internal SSD

[MacOG728893]

Would this work okay? I have a fast PCIe SSD that gets about 700MB/s read and write speeds. I got the 512GB version. I have a Windows 8.1 Pro partition for gaming that eats up about 180GB's. Because of that, my plan is to use my SSD as the "scratch disk" for my current projects because of it's obvious speed advantage. When I'm done with the project, I'll archive to my LaCie RAID 1 for backup.

I know everyone recommends working off an external, but even my LaCie in RAID 0 cannot match the speeds of my internal PCIe SSD. I'm just wondering if this method seems like a sure fire way to get top disk speed performance while preserving my projects once finished on my archive disk for possible later use?

If so, please let me know. If you have a better suggestion, shoot. Also, I am curious as to how you guys deal with file management and preparing a project to edit.

Thanks!

Edit: I found some great information in this thread, but it didn't really answer my specific question. I know it's "not good" to store your applications and media on one drive, but I thought that was because of the bottleneck of the drive. If 700MB/s isn't fast enough, what is?

 

[ppc_michael]

It will work, but will put more cycles on your SSD.

And honestly, if all your footage will fit on a 512GB drive, it is likely the footage itself has a bitrate well under the speed limit of a traditional hard drive, so you wouldn't encounter the bottleneck in the first place.

 

[MacOG728893]

It will work, but will put more cycles on your SSD.

And honestly, if all your footage will fit on a 512GB drive, it is likely the footage itself has a bitrate well under the speed limit of a traditional hard drive, so you wouldn't encounter the bottleneck in the first place.

Thanks for you the information. So what is your recommendation then?

 

[ChrisA]

I know everyone recommends working off an external, but even my LaCie in RAID 0 cannot match the speeds of my internal PCIe SSD....

This recommendations are from the days before there were SSDs. They still apply if you are using spinning disk drives

 

[MacOG728893]

This recommendations are from the days before there were SSDs. They still apply if you are using spinning disk drives

Okay, so there is no problem with me using my PCIe SSD as me scratch and boot drive then?

 

[nateo200]

Okay, so there is no problem with me using my PCIe SSD as me scratch and boot drive then?

Nope. Just keep in mind your increasing write cycles on your internal SSD which has limited write drives unlike spinning HDD's...of course it takes years to wear an SSD down with write cycles but something to keep in mind when you start writing terabytes+ a month from editing it will obviously speed up (still a long time though)...regardless you CAN upgrade the internal SSD, its not as straight forward as with the cMBP's but its doable.

In the future if I were you I would look at USB 3.0 or Thunderbolt SSD's, you can get some insane performance from using an SSD as a scratch drive + having an internal SSD, video projects that would require a RAID 0 with 2-4 drives are possible with my tiny Intel 520 SSD hooked up via USB 3.0...haven't tried Thunderbolt SSD's beyond basic tests but most SSD's (outside of Apples PCIe method) are going to be SATAIII which max's at 6gbps and USB 3.0 is 4.8Gbps...of course their is overhead with USB 3 and Thunderbolt might be faster but still...just some things to keep in mind.

 

[MacOG728893]

Nope. Just keep in mind your increasing write cycles on your internal SSD which has limited write drives unlike spinning HDD's...of course it takes years to wear an SSD down with write cycles but something to keep in mind when you start writing terabytes+ a month from editing it will obviously speed up (still a long time though)...regardless you CAN upgrade the internal SSD, its not as straight forward as with the cMBP's but its doable.

In the future if I were you I would look at USB 3.0 or Thunderbolt SSD's, you can get some insane performance from using an SSD as a scratch drive + having an internal SSD, video projects that would require a RAID 0 with 2-4 drives are possible with my tiny Intel 520 SSD hooked up via USB 3.0...haven't tried Thunderbolt SSD's beyond basic tests but most SSD's (outside of Apples PCIe method) are going to be SATAIII which max's at 6gbps and USB 3.0 is 4.8Gbps...of course their is overhead with USB 3 and Thunderbolt might be faster but still...just some things to keep in mind.

Thanks for the info! I have a LaCie Thunderbolt 2big Quadra 4TB that I just got for my birthday. I was planning on using that simply for archiving. I think I'll use the internal SSD for now and eventually use the free SATA port in my iMac to add a 2.5" SSD for my scratch disk, possibly the 1TB Samsung Evo.

Just to clarify though, the speed of my internal PCIe SSD is much faster than using the external HDD method for editing correct?

 

[nateo200]

Thanks for the info! I have a LaCie Thunderbolt 2big Quadra 4TB that I just got for my birthday. I was planning on using that simply for archiving. I think I'll use the internal SSD for now and eventually use the free SATA port in my iMac to add a 2.5" SSD for my scratch disk, possibly the 1TB Samsung Evo.

Just to clarify though, the speed of my internal PCIe SSD is much faster than using the external HDD method for editing correct?

You have a LaCie 2big Quadra? Jesus put that thing in RAID 0 with another cheapo 4TB drive for back up and you've got a beast scratch disk, I believe the Quadra is set to RAID 0 by default if so you have a fast drive (though not even half as fast as your internal SSD), problem with RAID 0 is if one drive fails all your data is gone, for the future I would look up RAID arrays and possibly other LaCie drive systems with 4 or 6 drive bays....but yeah the speed is above and beyond the external HDD method (unless we are talking external SSD's or HDD's in RAID 0/5 configuration which we aren't). Typical external spinning disks are like 60-110MB/s while a SATAIII SSD is 300-575MB/s and a PCIe SSD is 700-1250MB/s, I have a SATAIII SSD for both internal and external use and they are plenty fast enough, you have a PCIe SSD so you've got double the speed than me.

 

[ChrisA]

The wear down argument on the internal SDD is real but it might take longer to wear out the SSD then the typical life of a macbook. Maybe if you used the macbook every day for 20 years? Video editing does NOT write as much data as you might think. The video files are written once and all your edits are captured in a small XML file. The video data are read many, many times while editing but written once. So it would take tens of thousands of video edit projects to use up an SSD. Likely the aluminum unibody case will get re-cycled into coke cans before the SSD is word out.

Different story if you tried to build an e-commerce database on the macbook and used SSD a a write cache. But even in that case (yes LOTs of people use SSD a write caches) the system lifetime is shorted then the SSD lifetime.

We have lived with shorter lifetimes. A spinning disk is dead typically n five years, more or less. the SSD should last longer if all you are doing is video editing.


As for archives, I used bare SATA drives. with a device like this
http://www.amazon.com/Kingwin-2-5-Inch-3-5-Inch-Enclosure-EZD-2535/dp/B001HNO3T8/ref=pd_sim_e_4
then I put the drive in a box like this
http://www.amazon.com/IO-Crest-3-5-Inch-Storage-SY-ACC35011/dp/B003OBZQVU/ref=sr_1_5?s=electronics&ie=UTF8&qid=1385485158&sr=1-5&keywords=sata+box
and then the drive in the box goes into this
http://www.amazon.com/SentrySafe-H4300-Fire-Safe-Waterproof-Security/dp/B004QWZINW/ref=sr_1_1?ie=UTF8&qid=1385485264&sr=8-1&keywords=fire+safe

I don't keep old data from old project on-line. I make a couple copies and it gets "filed" in the safe.

 

[MacOG728893]

You have a LaCie 2big Quadra? Jesus put that thing in RAID 0 with another cheapo 4TB drive for back up and you've got a beast scratch disk, I believe the Quadra is set to RAID 0 by default if so you have a fast drive (though not even half as fast as your internal SSD), problem with RAID 0 is if one drive fails all your data is gone, for the future I would look up RAID arrays and possibly other LaCie drive systems with 4 or 6 drive bays....but yeah the speed is above and beyond the external HDD method (unless we are talking external SSD's or HDD's in RAID 0/5 configuration which we aren't). Typical external spinning disks are like 60-110MB/s while a SATAIII SSD is 300-575MB/s and a PCIe SSD is 700-1250MB/s, I have a SATAIII SSD for both internal and external use and they are plenty fast enough, you have a PCIe SSD so you've got double the speed than me.

Sorry to sound confused, but why would I use my LaCie external for my scratch disk if my SSD is at least twice as fast, even in RAID 0? What am I missing here?

The wear down argument on the internal SDD is real but it might take longer to wear out the SSD then the typical life of a macbook. Maybe if you used the macbook every day for 20 years? Video editing does NOT write as much data as you might think. The video files are written once and all your edits are captured in a small XML file. The video data are read many, many times while editing but written once. So it would take tens of thousands of video edit projects to use up an SSD. Likely the aluminum unibody case will get re-cycled into coke cans before the SSD is word out.

Different story if you tried to build an e-commerce database on the macbook and used SSD a a write cache. But even in that case (yes LOTs of people use SSD a write caches) the system lifetime is shorted then the SSD lifetime.

We have lived with shorter lifetimes. A spinning disk is dead typically n five years, more or less. the SSD should last longer if all you are doing is video editing.


As for archives, I used bare SATA drives. with a device like this
http://www.amazon.com/Kingwin-2-5-Inch-3-5-Inch-Enclosure-EZD-2535/dp/B001HNO3T8/ref=pd_sim_e_4
then I put the drive in a box like this
http://www.amazon.com/IO-Crest-3-5-Inch-Storage-SY-ACC35011/dp/B003OBZQVU/ref=sr_1_5?s=electronics&ie=UTF8&qid=1385485158&sr=1-5&keywords=sata+box
and then the drive in the box goes into this
http://www.amazon.com/SentrySafe-H4300-Fire-Safe-Waterproof-Security/dp/B004QWZINW/ref=sr_1_1?ie=UTF8&qid=1385485264&sr=8-1&keywords=fire+safe

I don't keep old data from old project on-line. I make a couple copies and it gets "filed" in the safe.

How come you do not keep the data from old projects? What if you needed to make an edit in the future? Sorry for all the questions, but setting a boot drive, scratch drive, proper redundant backups off and on site can get quite confusing, also expensive.

 

[nateo200]

Sorry to sound confused, but why would I use my LaCie external for my scratch disk if my SSD is at least twice as fast, even in RAID 0? What am I missing here?

Because one day you might run into a project that exceeds internal SSD space by allot...and thats assuming you have NOTHING on your internal drive. My internal drive has other applications, games, movies, etc. on it so anything other than small projects has a dedicated drive. What if your house wasn't big enough to comfortably hold all your stuff? Would you fill your house up to the max and make it uncomfortable to move around for you and everyone else or maybe build a larger garage or rent a storage unit?

How come you do not keep the data from old projects? What if you needed to make an edit in the future? Sorry for all the questions, but setting a boot drive, scratch drive, proper redundant backups off and on site can get quite confusing, also expensive.
Because it takes space...allot of it...I offload to Blu-ray disks for archiving in a high quality format, in fact I just made a thread about it asking for help with a specific codec but in general I archive in a high quality format so if I need to use I can still manipulate it pretty good.

read the bold, I typed in the bold.

 

[MacOG728893]

read the bold, I typed in the bold.

Smart move. It makes sense. Right now I shouldn't have anything over 150GB mark in terms of RAW footage, so I'll get by. However, I shall take your words to heart and move towards that direction as my projects grow!

 

[nateo200]

Smart move. It makes sense. Right now I should have anything over 150GB mark in terms of RAW footage, so I'll get by. However, I shall take your words to heart and move towards that direction as my projects grow!

Yeah I've gotten to the point where buying a 1TB Hard Drive is like filling a tank of Gas (ok not THAT bad but similar justification!)..now that I'm starting to get paid for photo and video jobs I'm not so stressed over it. However in about 1 or 2 more external Hard drives I'll probably looking at a RAID array since I'm up to my neck in data. Remember to organize your footage and rotate it among drives as well, I've had two drives fail on me, luckily the drives were just 8 years of DVD rips and not some poor clients whole day...that was a very bad day though needless to say!

 

[MacOG728893]

Yeah I've gotten to the point where buying a 1TB Hard Drive is like filling a tank of Gas (ok not THAT bad but similar justification!)..now that I'm starting to get paid for photo and video jobs I'm not so stressed over it. However in about 1 or 2 more external Hard drives I'll probably looking at a RAID array since I'm up to my neck in data. Remember to organize your footage and rotate it among drives as well, I've had two drives fail on me, luckily the drives were just 8 years of DVD rips and not some poor clients whole day...that was a very bad day though needless to say!

Yeah, I just setup my CrashPlan+ account. Here's the low down.

- 512GB PCIe SSD
> 250GB's Currently free in OSX
> The remainder is my 8.1 Pro partition for gaming
> The 250GB will currently be used as my scratch disk

- 4TB LaCie 2big Thunderbolt RAID1
> Backup and Archive for personal files and projects

- 1TB CalDigit AV Drive
> Time Machine Backup

- CrashPlan+
> Backing up iMac
> Backing up Archive

On a side note. I am quite disappointed with the performance of FCX over PPCS6. However, I am loving the new way of editing in it. Hopefully Apple brings out a new version of FCX next month!

 

[ChrisA]

Yeah, I just setup my CrashPlan+ account. Here's the low down.

- 512GB PCIe SSD
> 250GB's Currently free in OSX
> The remainder is my 8.1 Pro partition for gaming
> The 250GB will currently be used as my scratch disk

- 4TB LaCie 2big Thunderbolt RAID1
> Backup and Archive for personal files and projects

- 1TB CalDigit AV Drive
> Time Machine Backup

- CrashPlan+
> Backing up iMac
> Backing up Time Machine
> Backing up Archive

On a side note. I am quite disappointed with the performance of FCX over PPCS6. However, I am loving the new way of editing in it. Hopefully Apple brings out a new version of FCX next month!

Yes, It's on Apple's website: New FCPX in December. This is one of the few times they pre-anounced a release of FCP.

Back to speed. FCPX allows you to choose if you transcode the video footage on import. You can tell FCPX to convert it all to ProRes. This takes time but makes for faster editing IF and only IF you have a very fast storage system like SSD. Play with the import options to see which is faster.

I don't see a reason to backup your TM drive to Crashplan. It will just slow down everything. Better I think to backup the system and user files twice as frequently.

----------

Yeah I've gotten to the point where buying a 1TB Hard Drive is like filling a tank of Gas (ok not THAT bad but similar justification!)..now that I'm starting to get paid for photo and video jobs I'm not so stressed over it. However in about 1 or 2 more external Hard drives I'll probably looking at a RAID array since I'm up to my neck in data. ....

Once you go to RAID, backup gets even harder. The only way to backup a RAID is with a second RAID that lives in some remote location. You can't leave the second RAID near the first of the same fire, flood the equipment thief will take down the backup too. You can try Crashplan but it could take two months or more the first take you connect.

You are best off copying twice to those 1TB (or now 3TB) drives and rotating them between fire safes in the office and at home.

 

[nateo200]

Once you go to RAID, backup gets even harder. The only way to backup a RAID is with a second RAID that lives in some remote location. You can't leave the second RAID near the first of the same fire, flood the equipment thief will take down the backup too. You can try Crashplan but it could take two months or more the first take you connect.

You are best off copying twice to those 1TB (or now 3TB) drives and rotating them between fire safes in the office and at home.

Agreed thats one way of putting it or you can risk it or you can use Blu-ray/DVD backups or you can back up to 1TB SSD's in MILSPEC cases or you can back up online...my point is there are plenty of solutions to back up your data RAID or not, there are RAID set ups that use 3 drives for RAID 0 and one for redudancy so you'd have say three 240GB SSD's in RAID 0 and a 750GB HDD thats in the physical disk array but not officially part of the RAID to back up to...you could then back up that 750GB HDD onto something else and store it in a fireproof safe and if needed store online another copy or mail another copy of the back up to a reliable friend you know to store...I don't think worrying about back up solutions is something to worry about with RAID if you do it right. Personally whenever I've used RAID's I've made regular back ups to another drive, and never use them to store long term, they are sort of like a modern day RAM disk to me, of course slower but similar purpose. Anyways I'm getting a Drobo and whether I use it as just JBOD or RAID 0 with 1 back up disk I will decide...I just happen to have a handful of 2.5" HDD's laying around and a bunch of identical ones too.

 

[ChrisA]

Agreed thats one way of putting it or you can risk it or you can use Blu-ray/DVD backups or you can back up to 1TB SSD's in MILSPEC cases or you can back up online...my point is there are plenty of solutions to back up your data RAID or not, there are RAID set ups that use 3 drives for RAID 0 and one for redundancy so you'd have say three 240GB SSD's in RAID 0 and a 750GB HDD...

No, that was not what i meant. It is easy to back up a tiny SSD Raid that is less then 1TB. Just copy it a 1TB disk. Now there are 4TB drives

What I meant was that if you have a RAID with more than about 5TB or 6Tb of data stored in it, then it gets hard to back it up.

Just "risking it" is OK if you are doing this for fun. But if video is paying the bills you could be wiped out if lightening stikes the power pole down the street or if some thief things you RAID box looks expensive and takes it.

Optical disks make poor backups because you have to periodically verify they are readable. They have a finite life and WILL fail. I tried using them for a while and gave up and went to rotating a few SATA drives.

 

[monokakata]

A quick note .. . Amazon has the 3 TB Seagates for $99 but it's only for the next 10 or 11 minutes.

 

[MacOG728893]

I don't see a reason to backup your TM drive to Crashplan. It will just slow down everything. Better I think to backup the system and user files twice as frequently.

Very good point. It certainly makes more sense to backup my main drive rather than Time Machine. With this change in mind, how do you feel about my original plan?

Once you go to RAID, backup gets even harder. The only way to backup a RAID is with a second RAID that lives in some remote location. You can't leave the second RAID near the first of the same fire, flood the equipment thief will take down the backup too. You can try Crashplan but it could take two months or more the first take you connect.

You are best off copying twice to those 1TB (or now 3TB) drives and rotating them between fire safes in the office and at home.

What about using their seeded backup and then continue the rest through the internet? Seems like that might work for now.

I like the idea of having a second RAID1 at my office, however that is quite expensive. I may do that down the road, but one LaCie was pricey, let alone two.

 

[nateo200]

No, that was not what i meant. It is easy to back up a tiny SSD Raid that is less then 1TB. Just copy it a 1TB disk. Now there are 4TB drives

What I meant was that if you have a RAID with more than about 5TB or 6Tb of data stored in it, then it gets hard to back it up.

Just "risking it" is OK if you are doing this for fun. But if video is paying the bills you could be wiped out if lightening stikes the power pole down the street or if some thief things you RAID box looks expensive and takes it.

Optical disks make poor backups because you have to periodically verify they are readable. They have a finite life and WILL fail. I tried using them for a while and gave up and went to rotating a few SATA drives.

What I listed isn't a definitive plan...and yes I know about the quarells of broken hard drives, discs that don't read, etc. after >5 years and often less. But I see what you mean now...I don't think I would do RAID 0 or even RAID 5 with over 1 or two terabytes worth of drives, besides the enclosure I'm getting is empty, you have to buy your own drives which is great I think.

 

[MacOG728893]

What I listed isn't a definitive plan...and yes I know about the quarells of broken hard drives, discs that don't read, etc. after >5 years and often less. But I see what you mean now...I don't think I would do RAID 0 or even RAID 5 with over 1 or two terabytes worth of drives, besides the enclosure I'm getting is empty, you have to buy your own drives which is great I think.

I think RAID1 with 3 available disks rotating between home and the office is genius.