Can I use a 3 Drive internal RAID as a Photoshop scratch disk?

sngraphics · Oct 31, 2012

I am putting together a 3 drive RAID internally in my MacPro 2009 Quad 2.93GHz to store my working files.
Boot/Apps are on Accelsior.
The drives I am using are Seagate ST2000DM001.

What I was wondering is, if it is possible to use this 3 drive RAID as a Photoshop scratch disk as well?
Will Photoshop have any issues using a RAID volume as a scratch disk?
Will I see benefits as compared to using a single HD?
Any comments or opinions would be greatly appreciated.

wonderspark · Nov 1, 2012

Yes, no problem with that at all. I use my 8-drive RAID 6 as scratch for Photoshop, and have used an internal 3-drive RAID 0 in my Mac Pro as well.

deconstruct60 · Nov 1, 2012

sngraphics said:
I am putting together a 3 drive RAID internally ...
The drives I am using are Seagate ST2000DM001.

What I was wondering is, if it is possible to use this 3 drive RAID as a Photoshop scratch disk as well?

You can, but be aware that this set-up is 3x more likely to fail than a single HDD is. When it does your whole working area will be wiped out.

6TB of 'scratch' area is rather large ( it is more than what the Photoshop needs to hold what doesn't fit in RAM). If buying 3 2TB drives just so can pragmatically use the 'outer' 500GB of each for scratch storage then perhaps can get away with a much smaller back-up target than matching the total capacity. ( only ever going to to put 1.5TB on the RAID set so can use a slower 2TB drive for back-ups.)

If this is 3 not because of capacity needs, but primarily because " 3 > 2, so it will be faster" then probably need to re-examine the decision.

Will Photoshop have any issues using a RAID volume as a scratch disk?

Photoshop has no clue what the implementation is.

Will I see benefits as compared to using a single HD?
Any comments or opinions would be greatly appreciated.[/QUOTE]

wonderspark · Nov 1, 2012

I was under the impression that the 3-member RAID was for more than scratch, and that all sorts of working files would be in the volume as well. I did this for a while before I had my current system, and backed up everything on separate disks nightly or more. It can be effective if done right.

JavaTheHut · Nov 1, 2012

Short answer: Just get more RAM.

Long answer: What is your setup now for RAM, Size of files your working with, CS version, Are you getting pages out when you monitor it from Activity Monitor?

sngraphics · Nov 2, 2012

deconstruct60 said:
You can, but be aware that this set-up is 3x more likely to fail than a single HDD is. When it does your whole working area will be wiped out.

[/QUOTE]

Yes I understand this. Thank you.
That is why I will have multiple back-ups of this volume.
Time Machine, Clones, Off-Site etc.

Loa · Nov 2, 2012

Hello,

+1 to get more RAM. RAM for 2009+ MPs is very cheap, and will be a lot faster than HD, RAID0 and even SSD for scratch.

As for the RAID0, I'd partition my drives with a small 50GB partition, and RAID0 these partitions for the best performance. If you do it in Disk Utility, make sure that the 50GB partitions are on the top of the list.

Loa

deconstruct60 · Nov 2, 2012

Loa said:
+1 to get more RAM.

It is likely this really isn't Photoshop scratch only storage; it is just too large. It is probably more of a "Working projects" drive. Even if used one 1TB drive that is far bigger than any memory backing store. That said, Photoshop will work much better if it doesn't page portions of the open project out to disk in order to get work done.

As for the RAID0, I'd partition my drives with a small 50GB partition, and RAID0 these partitions for the best performance. If you do it in Disk Utility, make sure that the 50GB partitions are on the top of the list.

using 150GB out of 3,000GB for "speed" doesn't make much sense. In that context it would be far better just to buy a 240GB SSD for about the same amount of money. Extremely short stroking HDDs doesn't make much sense now that SSDs are reasonably affordable.

If overly concerned about Flash wear issues could partition that 240GB drive into 150GB (still same size) and unformatted 90GB area that pragmatically would over-provision the drive even more than the internal settings ( if the underlying controller has any smarts about leveling. )

sngraphics · Nov 2, 2012

wonderspark said:
I was under the impression that the 3-member RAID was for more than scratch, and that all sorts of working files would be in the volume as well. I did this for a while before I had my current system, and backed up everything on separate disks nightly or more. It can be effective if done right.

Yes. This RAID is being setup primarily to store my working files.
I have the open bays, I have the multiple drives and I thought, "why not use these to get a faster "Working Files" volume."
It should also help with faster backups.
My usual method of backup is cloning using CCC and I believe that with faster Reads the backups would complete in less time.
(I hope I'm not wrong!)
But, at the same time I also thought, "why not use the same volume as a Photoshop scratch?"
(I have been able to get speeds of 500MB/sec+ out of this volume and would really like to put that to good use)

One thing I forgot to ask is if I have to have a separate partition for the scratch or will it work just the same as using one volume for both?

Loa · Nov 2, 2012

deconstruct60 said:
using 150GB out of 3,000GB for "speed" doesn't make much sense. In that context it would be far better just to buy a 240GB SSD for about the same amount of money.

It makes sense if he uses the rest for other purposes. 150GB scratch volume, 2850GB work volume.

Of course he'd be better using a SSD for scratch(or better yet getting more RAM). But he was talking as if he already had the drives, so short stroking is the best solution given what he already has.

sngraphics: you *can* use the same volume for both scratch and work, but I would advise against it: it will create unnecessary file fragmentation on the volume, slowing everything down. Don't create a fast RAID only to use it in a manner that will slow it down...

Loa

sngraphics · Nov 2, 2012

Fragmentation. Short stroking the drives and RAIDing.

Loa said:
sngraphics: you *can* use the same volume for both scratch and work, but I would advise against it: it will create unnecessary file fragmentation on the volume, slowing everything down. Don't create a fast RAID only to use it in a manner that will slow it down...

Thanks for your responses.
About "create unnecessary file fragmentation".
Will using the volume for working files create the fragmentation or using it as a scratch create the fragmentation?
Or using the one volume for both?
And would the fragmentation affect the performance of Photoshop using it as the scratch?

About short stroking the drives and RAIDing.
Would I create the smaller partitions on each drive first and RAID them together to create
the Scratch volume and then RAID the larger partitions together to create the working volume?

OR

Would I create one large 6TB RAID (3x2TB) and then partition that volume into one small for scratch and one large for working files?

----------

JavaTheHut said:
Short answer: Just get more RAM.

Long answer: What is your setup now for RAM, Size of files your working with, CS version, Are you getting pages out when you monitor it from Activity Monitor?

I believe I have enough RAM.
I have 32GB (4x8GB) of OWC RAM.
I am using CS5 and have assigned 70% of the RAM to Photoshop in PS preferences.
My Photoshop files are about 500MB to a few GB.
If I have only one or two files open I do not get page outs.
But when I have many open (over 10) plus have other apps running then page outs begin and sometimes are pretty heavy.

One quick question, Could having a lot of Safari tabs open at once cause more page outs? (over 100 tabs)

Loa · Nov 2, 2012

sngraphics said:
Or using the one volume for both?
And would the fragmentation affect the performance of Photoshop using it as the scratch?

Yes, and yes.

sngraphics said:
Would I create the smaller partitions on each drive first and RAID them together to create
the Scratch volume and then RAID the larger partitions together to create the working volume?

Yes.

sngraphics said:
One quick question, Could having a lot of Safari tabs open at once cause more page outs? (over 100 tabs)

Probably! Never tried that many!

Also, the amount of page outs will give you a rough estimate of how useful your scratch disk will be.

Loa

deconstruct60 · Nov 3, 2012

Loa said:
It makes sense if he uses the rest for other purposes. 150GB scratch volume, 2850GB work volume.

Not really. There are two very significant problems here.

First, that "slow" 2850GB still faces the RAID-0 risk problem. So the 2850GB needs a relatively frequent and automated back-up. Rebalancing what would have been the RAID back up to working storage (and back-up ) will likely be more cost effective.

Second. anytime that "work" volume is accessed that will negate any speed advantage of the short stroked partition provides. It cannot be used for frequently accessed files. It can only be used for more archival data which is accessed only occasionally.

sngraphics: you *can* use the same volume for both scratch and work, but I would advise against it: it will create unnecessary file fragmentation on the volume, slowing everything down.

The only way it creates high fragmentation is if he fills either volume up to the brim with data. That shouldn't be done whether partitioning and/or debating SSD vs HDD.

davegoody · Nov 3, 2012

Eh ???

deconstruct60 said:
You can, but be aware that this set-up is 3x more likely to fail than a single HDD is. When it does your whole working area will be wiped out.

RUBBISH !!!!

Why is a 3 Disk RAID 3x more likely to fail than a single disk ?

3 disks set up as a STRIPE (RAID0) would, of course be more likely to fail, however, set up as a proper RAID has redundancy (one of the disks is redundant) so is therefore more that TWICE as reliable as a single disk ! - If you are going to give advice, it should be correct advice !

Loa · Nov 3, 2012

davegoody said:
RUBBISH !!!!

Read his words correctly. He was talking about a RAID0, and it is 3 times more likely to fail. Period.

Talking about parity RAIDs is another matter entirely, and nobody here was talking about them, since it's pointless for the OPs needs.

Loa

deconstruct60 · Nov 3, 2012

davegoody said:
RUBBISH !!!!

Why is a 3 Disk RAID 3x more likely to fail than a single disk ?

Because he didn't mention any PCI-e raid card or going through gyrations to connect any such card to the internal Mac Pro drive sleds. In that context, most likely talking about a software raid set up. Although RAID 10 is doable in software there are only 3 disks. Nor was it particularly likely he was talking about RAID-1 with a hot failure over drive. So yeah, I'm not trying to sell him a RAID card and presuming that he is talking about RAID-0.

3 disks set up as a STRIPE (RAID0) would, of course be more likely to fail, however, set up as a proper RAID has redundancy (one of the disks is redundant) so is therefore more that TWICE as reliable as a single disk ! - If you are going to give advice, it should be correct advice !

[/quote]

Parity RAID brings along additional failure modes so it isn't quite twice as reliable. And the non parity solutions require more drives (so more expensive).

Once add additional expensive of a "real RAID" card to the mix the SSD is far more likely to be effective if capacity isn't the primary driving issue.

sngraphics · Nov 5, 2012

davegoody said:
RUBBISH !!!!

Why is a 3 Disk RAID 3x more likely to fail than a single disk ?

3 disks set up as a STRIPE (RAID0) would, of course be more likely to fail, however, set up as a proper RAID has redundancy (one of the disks is redundant) so is therefore more that TWICE as reliable as a single disk ! - If you are going to give advice, it should be correct advice !

Loa said:
Read his words correctly. He was talking about a RAID0, and it is 3 times more likely to fail. Period.

Talking about parity RAIDs is another matter entirely, and nobody here was talking about them, since it's pointless for the OPs needs.

Loa

deconstruct60 said:
Because he didn't mention any PCI-e raid card or going through gyrations to connect any such card to the internal Mac Pro drive sleds. In that context, most likely talking about a software raid set up. Although RAID 10 is doable in software there are only 3 disks. Nor was it particularly likely he was talking about RAID-1 with a hot failure over drive. So yeah, I'm not trying to sell him a RAID card and presuming that he is talking about RAID-0.

Yes, the 3 drives will be setup as a RAID 0 using Disk Utility.
Yes, I understand that there IS 3 times more chances that something could go wrong.
I am not setting up any other RAID config.
I just will be very diligent about my backups as I already am. Just more so with a 3 drive RAID.
I am holding off on SSDs until new MacPros come out.
For now just using what I have to get a little more speed out of my current equipment.
eg. faster backups, when the need arises faster scratch, faster reads/writes for working files.
If I can accomplish these (and more) then that's great.
If there is anything else I am missing or I need to do to set this up correctly, I'm all ears!
I wanted to thank everyone for taking the time to write. Excellent comments & advice.

DanielCoffey · Nov 6, 2012

sngraphics said:
I am holding off on SSDs until new MacPros come out.

Why? Buy a SATA-III SSD now and enjoy it then transfer it to a 2013 (or whatever) Mac Pro and enjoy it even more.

sngraphics · Nov 13, 2012

DanielCoffey said:
Why? Buy a SATA-III SSD now and enjoy it then transfer it to a 2013 (or whatever) Mac Pro and enjoy it even more.

Already have an Accelsior as a Boot/Apps drive.
The Max speeds of the HDs I have are close to 200MB/s anyway.
And the most important thing is I already have them!
Don't want to spend more until I see what Apple does with the MacPro next year.
RAIDing these drives should get me by fine for a while.

thekev · Nov 13, 2012

deconstruct60 said:
Second. anytime that "work" volume is accessed that will negate any speed advantage of the short stroked partition provides. It cannot be used for frequently accessed files. It can only be used for more archival data which is accessed only occasionally.

.

It may depend partially on available ram, but photoshop tends to access scratch drives when saving large files due to compression being used by default. It gained a background save function, but this would still be annoying if the goal is maximum performance. An ssd seems like a simpler scratch drive. Ridiculous amounts of ram combined with allocating the majority of it to PS would also solve the problem within this single application. CS5 and on under OSX use all assigned memory.

sngraphics · Nov 26, 2012

deconstruct60 said:
Not really. There are two very significant problems here.

First, that "slow" 2850GB still faces the RAID-0 risk problem. So the 2850GB needs a relatively frequent and automated back-up. Rebalancing what would have been the RAID back up to working storage (and back-up ) will likely be more cost effective.

Second. anytime that "work" volume is accessed that will negate any speed advantage of the short stroked partition provides. It cannot be used for frequently accessed files. It can only be used for more archival data which is accessed only occasionally.

So short stroking the drives to get a smaller volume just to be used for a scratch will not show any
speed benefits when the larger volume is accessed?
If the "Scratch" & "Data/Working" partitions are seen as 2 distinct volumes why would this happen?

JavaTheHut · Nov 26, 2012

sngraphics said:
So short stroking the drives to get a smaller volume just to be used for a scratch will not show any
speed benefits when the larger volume is accessed?
If the "Scratch" & "Data/Working" partitions are seen as 2 distinct volumes why would this happen?

Hi, Since the partitions A & B are on the same 3 physical striped disks the data is being requested at the same time creating a bottle neck of data. So "Scratch" is trying to be read at the same time as "Data/Working" is trying to be written. 2 things are being attempted at the same time.
You could still Partition the outer region of the RAIDed set to a fast work area. This would prevent the RAID set slowing down as "Data" fills up.

It would be recommended if you are unable to to address more RAM to have your Scratch on another SSD even a small one say 60GB. or a Fast HDD.

Ideally more RAM would prevent "Page Outs" from occurring. Also set your Photoshop: Preferences->File Handling->File Compatibility= Check "Disable Compression of PSD and PSB Files" & Set "Maximize PSD and PSB File Compatibility: to Never". This saves some time.

deconstruct60 · Nov 26, 2012

sngraphics said:
So short stroking the drives to get a smaller volume just to be used for a scratch will not show any
speed benefits when the larger volume is accessed?

If a disk is split into two logical volumes (partitions) if you are accessing both partitions then by definition you are not "short stroking" the outer one.
Moving the disk head out of the "outer track" volume and into the inner volume (to read/write ) will impact the access times to data in the "outer track" volumes. It is a physical constraint that short stroking is trying to avoid. A general HDD has the following elements.

or image http://en.wikipedia.org/wiki/File:Hard_drive-en.svg (from article
http://en.wikipedia.org/wiki/Hard_disk_drive )

Moving the head around is what drives up latency. By restricting the head movement only to the outer tracks it doesn't have to move as far. Shorter movements means faster average access times. If access data from anywhere on the disk then the head movements are the same as if there are no partitions/volumes on the same drive.

Blowing away the short stroke advantage by striping over RAID-0 doesn't really buy much back in most general cases on non-trivial sized files.

Partitioning Benefits/Weaknesses

" Benefits
....
Short Stroking", which aims to minimize performance-eating head repositioning delays by reducing the number of tracks used per hard drive
...
Weaknesses
...
Reduces overall disk performance on systems where data is accessed regularly and in parallel on multiple partitions, ... "
http://en.wikipedia.org/wiki/Disk_partitioning#Benefits_of_multiple_partitions

So even if just grabbing some small meta data from the "inner track" volume ( 32K of data) and most of the data from outer tracks ( 10,000K of data) merely diverting for the 32K kills the outer access average times.

sngraphics · Dec 3, 2012

thekev said:
It may depend partially on available ram, but photoshop tends to access scratch drives when saving large files due to compression being used by default. It gained a background save function, but this would still be annoying if the goal is maximum performance. An ssd seems like a simpler scratch drive. Ridiculous amounts of ram combined with allocating the majority of it to PS would also solve the problem within this single application. CS5 and on under OSX use all assigned memory.

I have 32GB of RAM installed and 70% assigned to Photoshop.
I would love to assign it all but from what I have been reading it might not be a good idea.
Need to leave some for system and other apps.
I have an SSD but it is only 60GB and when I am using very large files the amount of scratch needed usually goes well past this.
That is why I started this thread.
Because I am doing the 3 drive RAID for my working files anyway can this RAID double as a scratch on ocassion?

Most of the time I do not get "Page outs/Swap Used".
When I use very large files 3-4GB each and a few at the same time then it goes to the scratch. Which is really not that often.
Usually when I am doing this type of Photoshop work I would not be accessing the RAID for anything else. eg saving or opening working files.
I am usually doing multiple things when I am not doing heavy Photoshop work.

thekev · Dec 4, 2012

sngraphics said:
Most of the time I do not get "Page outs/Swap Used".
When I use very large files 3-4GB each and a few at the same time then it goes to the scratch. Which is really not that often.
Usually when I am doing this type of Photoshop work I would not be accessing the RAID for anything else. eg saving or opening working files.
I am usually doing multiple things when I am not doing heavy Photoshop work.

Pageouts/swap used are different than scratch disks. They work in a similar manner, but scratch is application specific. Yeah 3-4GB is big. Not many people on here get quite that big, so they can usually fit it in ram. I just emphasize memory as it tends to be faster and often cheaper these days. The RAID could work, but it will slow down on saves if you're using it for storage as well as photoshop will be writing scratch data when it goes to compress the images (enabled by default). Worst case scenario spotlight will slow down the process even further if it's set to keep track of the raid.

Can I use a 3 Drive internal RAID as a Photoshop scratch disk?

macrumors member

macrumors 68040

macrumors G5

macrumors 68040

macrumors 6502

macrumors member

macrumors 68000

macrumors G5

macrumors member

macrumors 68000

macrumors member

macrumors 68000

macrumors G5

macrumors 6502

macrumors 68000

macrumors G5

macrumors member

macrumors 65816

macrumors member

macrumors 604

macrumors member

macrumors 6502

macrumors G5

macrumors member

macrumors 604

Our Staff