Best way to configure 2011 27" iMac for PS

[clg763]

I have done a considerable amount of searching on this and while I have found a lot of people with very similar needs, none quite seem to match up with mine. First of all, this computer is used almost solely for editing photos in my wife's photography business. The specs currently are:

27" late 2011 iMac
Core i5 2.7ghz quad
radeon 6790m gpu
20GB total memory (2x8gb + the 2x2gb that shipped with the machine)
OCZ Vertex 3 240GB SSD installed internally in the extra SATA port
1 TB HDD (original from mac)

As you can tell, I have started to upgrade the machine to perform better for my wife's business. Currently, the OS and apps are on the SSD and I have the 1TB HDD with everything else on it. My question has to do with the location of the scratch disk, her active file storage and backup storage drives. Ideally, I would have 3 internal drives, one for OS + Apps, one for scratch and one for active files. I understand that it is important to spread the load across all three to optimize performance, however, my internal SSD drive is quite a bit faster than the internal HDD and lightyears faster than an external usb or firewire drive.

Due to the fact that she wants to keep her internal DVD drive operational, I cannot install another internal or make an external SATA drive. So, what is the best way to handle all of this without blowing a fortune on external thunderbolt storage?

I purchased the 2x8GB of ram because I had hoped to install the full 32GB and use about half of it as a scratch disk but now I have learned that the scratch needs to be about 1/3 the total system memory or 11GB. Is that large enough? I have not been able to evaluate her scratch usage because she has not changed her process to take advantage of the new machine yet, she doesn't do any panorama work but she some of her steps would improve if she could open quite a few files at once, (15-30).

Also, is there any advantage to using a ram disk for scratch if the OS X unified buffer will store the scratch in the ram anyway? I have already noticed her 20GB maxed (including inactive memory) does that mean that it is caching her current scratch?

Where is the best location for her scratch and active files with the drives I have installed? I have been using her 1TB for active files and scratch and the SSD for OS and apps, is this best or would things speed up if I moved one of those over to the SSD? Would I notice a big enough speed increase to justify the purchase of external thunderbolt storage? I don't like the cost of these add-ons but if the performance will be increased substantially, I will go ahead and pursue it.

Lastly, what do you believe is the best way to handle back-up storage? I had been thinking about using a 4 drive raid enclosure filled 3TB HDD's for 6TB of total storage, is firewire fast enough or should I find a way to make thunderbolt/eSATA work?

My dream setup would be to purchase the LaCie Thunderbolt to eSATA adapter and attach a large eSATA raid enclosure for backup and then daisychain this to the seagate thunderbolt to eSATA 2.5" adapter with an SSD attached for the scratch disk. However, this arrangement will cost over 1k and I'd rather not spend that much if I can help it.

 

[talmy]

I'll only address backup. Backup is there for recovery of deleted files, failure of system drive, or theft and disaster recovery. You have 1.25 TB potentially to back up (you didn't mention having any other drives). Optimally you should have two external drives of at least 1.5TB capacity each, and, frankly, USB 2.0 will do.

Alternate between the drives for daily clone backups using SuperDuper! or CarbonCopyCloner doing smart backups so that only changed or new files are copied. This will be fast, even with USB 2.0, no matter how prolific your wife is.

Keep the drives off-site.

You can use more drives if convenient. The important points are:

1. RAID will not offer additional security. You get better redundancy from having multiple backup drives (and housings).
2. You must keep backups off-site
3. Having more than one backup stored per physical device is false security.
4. The smart backups are so fast that slow drives are not an issue. This is not like trying to back up a terabyte to DVDs or tape!

 

[clg763]

Thanks for the advice, so you are saying that a high speed connection for the backup drive is not necessary due to how smart the software is now, that makes sense. How do most people keep their backup off-site? I am aware of cloud back-ups but those solutions are expensive, is the only way to do this by physically picking up the drive from the off-site location, hooking it up to the computer and then transporting back to the other location every time you back up?

The amount of active storage she needs is quite a bit less than 1.25TB, more like 500GB on a given day, she keeps her recent files and basic media on the computer but archives the rest of her work onto DVD-R's. She has almost 2TB of archived work and it just grows.

 

[talmy]

If one has two locations available, then use the alternate location. For instance I have a day job but work at home as well, so I keep my home backups at my day job location. There's an IT staff that keeps the daily (day job) backups at their homes.

Those archival backups (on DVDs) need to be at least in duplicate as well. The DVD media does degrade over time and should be recopied every five years or so to be safe.

 

[clg763]

Yeah, I know all too well how long DVD R's last, we re-burn them every few years but will probably switch to a raid array soon, I think she has a few hundred DVD's and that is only going to get more time consuming.

Still no bits on how to optimize the performance? I may try this over at the software forum to see if there are any takers over there.

 

[talmy]

I've got no suggestions on optimizing performance because I've never had any performance issues with it. I don't deal with images large enough to have swapping issues. If anything, for me it is CPU limited and you can't replace your i5 with the i7. I know I've never bumped against the 8GB I've got, although I'm always using over 4. I try to keep within an Aperture flow, dropping into Photoshop CS5 only when I need some feature Aperture doesn't have or doesn't do well. Both apps run simultaneously, so there's that additional load.

The new Photoshop CS6 seems significantly faster. A software investment (when it comes out) might give more bang/buck than more hardware.

 

[Bear]

...
My question has to do with the location of the scratch disk, her active file storage and backup storage drives. Ideally, I would have 3 internal drives, one for OS + Apps, one for scratch and one for active files.
...
Also, is there any advantage to using a ram disk for scratch if the OS X unified buffer will store the scratch in the ram anyway?

With the amount of ram you have and the fact that your system disk is an SSD, I would just keep the scratch disk on the SSD. You would need another small SSD to notice any improvement over that. And it would be pretty small in most cases.

...
Lastly, what do you believe is the best way to handle back-up storage? I had been thinking about using a 4 drive raid enclosure filled 3TB HDD's for 6TB of total storage, is firewire fast enough or should I find a way to make thunderbolt/eSATA work?
...

Firewire is fine for a Time Machine drive. The only part that will take a noticeable while is the initial backup.

 

[mactmaster]

You should not use RAM disk for scratch. Scratch should be much larger since it will be used in cases when RAM is in low levels. I would also recommend to use your systems SSD as the scratch.

 

[clg763]

Thanks for the feedback, I think it really makes sense to use the SSD for scratch since it is so much faster than the 1TB HDD. I had been hearing a lot of people talking about how important it is to keep the locations of those things separate above all else which would not make sense if one drive was so much faster.

It sounds like we may move forward with a firewire archive system, probably keep two of them to stay safe, thanks for the advice, this helps me move forward with better clarity. Since I don't plan on using RAM for scratch space, does it make sense to upgrade her to full 32GB at this time (it will cost me $104 for two more 8GB sticks)? I have maxed out the RAM without using any of it for scratch yet, but half of that was inactive memory.

 

[theSeb]

I have maxed out the RAM without using any of it for scratch yet, but half of that was inactive memory.

Which does mean not that you maxed out the RAM. What were the page ins and page outs numbers?

 

[clg763]

Which does mean not that you maxed out the RAM. What were the page ins and page outs numbers?

It just means that I had no more free, there was 400MB page outs and around 50MB page ins, iirc.

 

[theSeb]

It just means that I had no more free, there was 400MB page outs and around 50MB page ins, iirc.

Free RAM is wasted RAM. OSX uses as much RAM as it sees. The fact that you had a lot of inactive RAM does not mean you has no more "free" RAM.

 

[Bear]

It just means that I had no more free, there was 400MB page outs and around 50MB page ins, iirc.

The Page outs is the critical numbers and it depends on over how much time the pages outs have accumulated.

It's a tough call as to whether you should upgrade the remaining ram. Since it's so easy to do, save it until you have a better idea on the performance?

Also, keep the original Apple ram in case you need warranty/AppleCare service on the system. For some issues, Apple wants the original memory in the Mac. This is usually for certain hardware issues. Also, if they decide to swap the system, you would want to hand them back the original memory.

 

[clg763]

The Page outs is the critical numbers and it depends on over how much time the pages outs have accumulated.

It's a tough call as to whether you should upgrade the remaining ram. Since it's so easy to do, save it until you have a better idea on the performance?

Also, keep the original Apple ram in case you need warranty/AppleCare service on the system. For some issues, Apple wants the original memory in the Mac. This is usually for certain hardware issues. Also, if they decide to swap the system, you would want to hand them back the original memory.

I think I'll hold off then, her old mac pro only has 4GB; I was originally just going to get her another 8GB which would have given 12 in this machine and even that would have been a massive upgrade.

 

[-hh]

For general performance enhancement stuff, I'd recommend checking out some of the advice from Digital Lloyd, such as his Mac Performance Guide.

I'll only address backup. Backup is there for recovery of deleted files, failure of system drive, or theft and disaster recovery. You have 1.25 TB potentially to back up (you didn't mention having any other drives). Optimally you should have two external drives of at least 1.5TB capacity each, and, frankly, USB 2.0 will do.

Alternate between the drives for daily clone backups using SuperDuper! or CarbonCopyCloner doing smart backups so that only changed or new files are copied. This will be fast, even with USB 2.0, no matter how prolific your wife is.

What I/O speed is adequate will depend on workflow. For using just Time Machine in the background, I'd generally agree....but for a workflow that includes making a full (not incremental) dump so that one can take the drive off-site ASAP, USB is going to be misery..

Keep the drives off-site.

Agreed, at least some of them. For a "Home" machine, one's desk drawer in the office at work can suffice; I've also had people suggest safety deposit box down at the bank. The Catch-22 is that the less convenient the off-site repository is, the less frequently it is probably going to be used...simple human nature.

For cloud storage/backup considerations, don't forget the bottleneck of your local ISP (besides usage caps) and run the math based on your local connection speed to see just how many hours-days-weeks it will take for however many GB you typically have when you come back from a photo sheet. For example, 100GB at a T1's 1.55Mbps = 155 hours = 6.5 days.

You can use more drives if convenient. The important points are:

1. RAID will not offer additional security. You get better redundancy from having multiple backup drives (and housings).
2. You must keep backups off-site
3. Having more than one backup stored per physical device is false security.
4. The smart backups are so fast that slow drives are not an issue. This is not like trying to back up a terabyte to DVDs or tape!

My two cents are:

(Yes, always have more backup copies than you think you'll need)

1. RAID 0 is performance, not safety. RAID 1 isn't that expensive to add, nor a particularly bad idea for one's "First Tier" local backup. If employed, make sure to use it with a configuration that proactively notifies you when it goes bad. Ditto for any consideration of a NAS.
2. Yes, some backups should most definitely be kept off-site, in case your home/business burns to the ground, flooding, etc. But also be careful of creating an "All Eggs in One Basket" single point of failiure opportunity at the remote site. In other words, keep copies both locally & remote.
3. Agreed.
4. True, but this mostly only applies to incremental backups. Full (virgin) backups and copies should be part of the mix too as another risk management contingency, and this is where a slow interface will turn this into an hours-long chore which makes it less likely due to human nature to be used as regularly as it should.

With the Mac, there's a couple of options. For example:

Local - Disk A: Time Machine ... incremental backups
Local - Disk B: RAID 1, OS/Marco based backup (doesn't use TM)
Local- Disk C: Carbon Copy Clone of Disk A
Remote - Disk D: Carbon Copy Clone of Disk B
Remote - Disk E: As D; but a prior backup that's used in disk rotation not to be reused until other backups have been verfied as operational
Local - Disk F: As E and third disk in a three disk rotation.

For I/O performance:
A - USB is OK
B - USB is OK
C - same as A
D - probably want faster than USB
E - as D
F - as D

Since C stays local, it can be a standard external drive.

Since D/E/F get move to remote, and the "move to" can be a dreaded chore, having faster I/O here helps minimize that chore quotient. Day to day expenses can be reduced some by using bare drives and what I personally call a "Toaster" Hard Drive Dock style of interface. The "VoyagerQ" from NewerTech is an example and comes with FW/USB/eSATA and is only $80, which basically means it pays for itself after ~4 cheap external drive enclosures.

-hh

 

[talmy]

True, but this mostly only applies to incremental backups. Full (virgin) backups and copies should be part of the mix too as another risk management contingency, and this is where a slow interface will turn this into an hours-long chore which makes it less likely due to human nature to be used as regularly as it should.

I've personally never had any problems with these smart backups, which aren't really the traditional "incremental backup" since you end up with a clone and not the incremental changes. I've had problems in the past with incremental backups in tape backup environments.

But this does bring up an important point -- regularly verify that the backups are working by doing random restores or booting of clone system drives. I've been burned in the past when I've discovered that backups were empty or otherwise non-restorable, or that "automatic" backups were not actually being performed! The latter seems to be a frequent occurrence when relying on other people (or IT departments) to do the backups for you.