Backing up to large media

[Lastmboy]

I'm planning to buy an iMac shortly (hoping the new ones come out soon). I plan to have a SSD system drive with a 12 TB Thunderbolt RAID 5 array for data and other storage. I already have a 12 TB USB 3.0 RAID 5 array that I would like to use as a backup drive.

Here's where the confusion comes in. As I understand it, with TimeMachine, I can only set up one profile. Therefore, if I set it to do an entire backup of my system drive, then I can't also set it to back up parts of my data array. Also, from what I understand, TimeMachine requires a complete drive dedicated to it.

Therefore, some questions:

1) How can I take full system backups of my SSD system drive, and also take incremental backups of changed data on my TB RAID array?

2) Can I use my 12 TB USB 3.0 array for backups? Do I have to either dedicate the entire array to TimeMachine backups or buy a separate, smaller external drive just for TimeMachine?

3) Can I somehow partition the RAID array to use some of it for TimeMachine backups, but leaving the rest available for other data or backups?

4) Is there other software that would work better for this situation?

5) If I have Bootcamp set up for Windows, does TimeMachine or other software backup that partition too?

Thanks

 

[chickenwingfly]

Hello.

This will only be a half-assed help reply, but, with CarbonCopyCloner (free, you can donate though), there is a lot that can be done (hence why I decided to use it instead of TM). Now, I'm STILL in "production" stages, so this backup plan that I'll describe is not fully deployed, but will in the next weeks, before I upgrade to ML.

Notice that, with CCC you can have profiles for several backup "tasks" so you can have as much as you want (even for individual folders).

Premises:
1. I have a main disk, internal (OSDisk).
2. I have an external disk with all data (Data1Disk) and a partition equal to OSDisk (so the space for data is Data1Disk - OSDisk).
3. I have an external for backup of them all (Data2Disk).

Plan:
1. Backup OSDisk to Data1Disk.
2. Backup Data1Disk to Data2Disk.

So at any time I have three bootable disks (internal, Data1Disk containing a copy of OSDisk and Data2Disk containing this copy as well) and redundancy of my data (via Data2Disk).
And as you can see I can use Data1Disk as the source of my "big" media and also as a destination for backing up the internal OS disk.

As for Bootcamp, this is what the FAQ says.

Notice I'm in no way affiliated to CCC or anything, I just find CCC to be one of the best apps I ever used and their user support (which I think is a one-man solo attempt) is top notch (hence why they deserve a donation hands down).

 

[Lastmboy]

Thanks for the info. I'll check out CCC.

 

[flynz4]

I use CCC for part of my backup plan (a small part). I think it is a great program and I gladly donated.

I suggest that you seriously consider a comprehensive plan. For me, it consists of two key elements... and also an extra third "just in case" element. My rationale follows. You needs might be different.

First and foremost, I need an automatic offsite backup as the #1 priority. All of the backups in your house will not help if they are all stollen, or if you house is burned. I personally use Crashplan+. I am backing up 1.5 TB of primary data to CP+ automatically... and unlimited backup is only $3/month.

Second: I very much want Time Machine. I back up the vast majority of my data to a 3TB Time Capsule because it just works. The user interface of TM is just spectacular... and the usability is fantastic if you ever need to rebuild a computer or upgrade a computer (something that I do often)

The above two items give me full backup of everything both on site (for ease) and offsite (for disaster recovery).

The third thing that I do is move just my personal media (and only non-identity theft type of data) to a pair of external 1.5TB HDDs, and keep one copy locked in my desk drawer at work. The media includes an Aperture Vault, my entire Aperture library, and all of my personally created videos (ex: home camcorder videos). I do this because this data is totally irreplaceable. I use CCC to keep this up to date. Any time I have a significant change to my media collection (ex: returning from a vacation with new photos)... I attach the HDD already at home... perform an Aperture Vault update, then perform a CCC incremental sync... and a few minutes later I have an up to data copy that I bring to the office and swap with the 2nd HDD locked in my desk.

The reason that ONLY keep non-identity theft information on that drive is because of the decreased security of such a backup copy.

In terms of security... highest to lowest... the 3 backups are:

1) Crashplan+ - Extreme Secure
2) Time Machine - relatively unsecured... but I do keep my TC locked in a separate location in the house
3) HDD in a locked desk drawer at work (least secure)

I do not know what type of data you have on your large array. I also have a Thunderbolt array (8TB Promise Pegasus R4). I love it... it absolutely screams.

For me... I have little to no reason to back up easily obtainable consumer media such as movies etc. I do keep my full music collection backed up, but mostly because that is relatively small (about 100GB). That is backed up to both Crashplan+ and TM/TC.

I do not even try to backup my movie collection other than inside my house (to a separate NAS)... but that will be moving to a dedicated Apple server at some point. My reason is that a movie collection is too large to affectively store to Crashplan+... plus it is pretty trivial to re-rip if so desired. Also.. if my house burned down and all the DVDs were destroyed, they would be covered by insurance. Bottom line: Unlike pictures, camcorder videos, etc... commercial movies are easily replaced. Unlike music... commercial videos offer backup challenges because of slow internet speeds.

Finally, I am not a fan of keeping bootable backups around unless you have a very specific need to be up and running immediately. They do not really help that much if you are upgrading equipment because they have the wrong drivers etc. The biggest issue is that keeping an up to date clone drive usually destroys all history of changes at each backup (or stores them helplessly away in an archive directory). If it comes down to needing to rebuild a computer... I would rather just buy a new computer, and then restore users, application and data from a TM backup. That works flawlessly even if you upgrade a computer model in the process. Hence for me... a bootable clone has little value.

Hope this helps.

/Jim

 

[Lastmboy]

Thanks for taking the time to share your backup strategy. That's pretty much what I'm after, and am part way there.

I currently use Carbonite online backup service for my "critical" files. That updates on the fly, so I know I always have the latest version in the cloud. It works good, but is cumbersome to setup. I went and checked out Crashplan+, and it looks like it could be better for less money. I'll definitely consider that. Do you not have any issues storing large amounts of data with them? I've never run into the problem, but have been told that Carbonite will throttle me way back if I exceed a certain threshold. I only have about 50GB on it, but with incremental backups and versioning, that can add up over time. I would prefer to store more like 1.5TB (like you), but figured that might limit my service. It surprises me how Crashplan says you can store unlimited data, and that they keep all versions forever. Wouldn't that start to get costly for them over time?

I don't backup any movie rips at all, as, like you say, they could be recovered in the event of an emergency. Most of my music is in iCloud via Apple's Match service. However, I have a fairly large library of uncompressed music that isn't backed up. It is, or course, possible to get that back with some work.

My photo library scares me the most, as I don't have an offsite backup of it. I do keep a local backup, but that won't help in the event of theft, fire, etc. However, if Crashplan actually lets you store unlimited data, my library would likely fit on there. It's currently about 200GB. Can you setup Crashplan so that it will pick up new files/photos as they are added to your primary drive?

I'm still on a pc, desperately wanting to get going on an iMac. I currently backup my entire system drive (160gb SSD) weekly to an image on a NAS box. I was hoping to do something similar with the iMac. I know TimeMachine does that, but what I didn't want was to have it lock up my entire RAID array to backup a 160gb drive. Do I need a separate external drive formatted just for the TM system backup?

Glad to hear that you like your Promise Pegasus. I've talked myself into forking out the cash for one of those. I've seen videos of it in action, and it really flies. My concern was how do I use TimeMachine to backup both my system drive and some of the data I would have on the Pegasus. Do I use TM for one and CCC for the other? Does CCC require a dedicated drive to backup up to, or could it backup to a folder on a RAID array? I guess it comes down to whether you need a bootable backup or not, eh?

I don't have the security requirements that you do, as all backups will be done at my main office at home. However, I like all of your ideas. Just have to sort out what to put where. The biggest issue for me was the critical files. One of the online services seems to work well for that, and Crashplan+ looks like it might be the best. If I could backup my photo library to the cloud, that would take care of my next biggest concern. The remaining files could be recovered with effort, but I would like to keep the work related stuff on a separate drive so I know I can at least recover them quickly.

Now, I'm just not sure what kind of setup I need for the local backup destination. Will my USB 3.0 RAID array work for multiple backup profiles, or do I need a separate device for basically every "profile" I setup. I guess another alternative is to just let TM backup everything to the USB 3.0 array, all in one blast. With using "live" backups, do you find it drags down your system quite a bit? I like my critical files backed up live, as I required the latest version and don't want to miss any. However, the other stuff could be backed up during off-peak hours.

Thanks again for your time and ideas. Very useful.

 

[flynz4]

You have lots of questions in there:

Crashplan really is unlimited, and they include unlimited versioning. The versioning is extremely important if you inadvertently delete something. I have talked to people who (by mistake) deleted an entire year of photos and did not notice it for a very long time. With CP+, you can just get it all back. I have mine set to back-up at 15 minute intervals. It keeps track of changes real time, and backups up at whatever interval you define. You will never notice it happening in the background.

Where you might have problems is with your ISP. When I had Verizon FIOS, we had unlimited data bandwidth. Now that Verizon has left my area, I switched to Comcast which has a 250GB/month data cap. My original data was already fully backed up, so when I switched to Comcast, I just had to deal with incremental updates (every 15 minutes) which is negligible. If I was starting from scratch now (and I will be when I bring a new primary computer online as soon as the new iMacs are introduced)... I'll pay the $120 or whatever it costs to do a physical seed via HDD with CP. They send you a 2TB drive... you back up to it locally (encrypted on your own machine)... and then fed-ex it back to them. You are instantly fully backed up without using any of your ISP bandwidth.

When I originally did my first backup (using FIOS), it took a month or so... but even then performance impact was not noticeable. I turned on data in a priority order based on importance and size. I started with just documents... which took a few hours. Then I added photos... which took maybe a week. By taking that approach, I was sure that my irreplaceable photos (as an example) were fully backed up before I started backing up replaceable music. Eventually... all users on the iMac were completely backed up.

Regarding the Pegasus... so far I have only been playing with it. For example... I copied my Aperture 3 library to the Pegasus, and then ran A3 on my MacBook Air off of the Pegasus. It flied, and was actually faster than running A3 on my i7 iMac. The Pegasus will not go into daily use until I get the new iMac. My wife and I currently share a single 2009 i7 iMac. Once I get the new machine, we will each have a dedicated machine and the Pegasus R4 will be used for high performance libraries such as Aperture 3. The data on the Pegasus will be automatically backed up every 15 minutes to CP+, and every hour to TM/TC. With Apple's TM, you do not get to control the backup interval which is just fine. Usually simpler is better.

Both TM and CCC will write to any volume and that can be shared with other data. I personally prefer dedicated drives for backup. Drive performance for backup just doesn't matter much. All of the backup programs that I use are incremental... so only the data that changes is what gets backed up. I currently use Time Capsules elsewhere on the home network (hard wired). That allows me to locate the TC's away from the computers, so if the local junkie does a smash & grab... he will not get both the computer and the local backup.

My photo library is larger than yours, and I have mine fully backed up locally (TC), to the cloud (CP+) and finally with CCC to a pair of cheap 1.5TB Seagate GoFlex drives that I rotate to the office. My photos are my most valuable digital assets. Rotating the media to my office is the only manual operation that I do. I generally do not recommend having any part of your backup strategy rely on human intervention. The only reason I do with my media is because it is extremely important to me, and primarily... because I already have dual automatic backup (TM & CP+) already in place.

If you are new to Macs... you owe it to yourself to try Time Machine. It really is a great program, which is dirt simple and it just works. It comes in very handy when you want to upgrade computers. For my usage... it is a few orders of magnitude more important than cloned backups. YMMV.

I'm sure I missed some of your questions... but hopefully this helps.

/Jim

 

[Lastmboy]

Thanks alot.

Bandwidth isn't an issue with me, as I have unlimited. In fact, I just discovered that they're installing a new fibre optic network with vastly improved speeds (still unlimited). Top speeds are 30Mbps upload and 200Mbps download, which will be very nice.

I'm going to switch to Crashplan when my Carbonite plan expires. I like everything I read about it. The apparently do not throttle, like Carbonite does. The seed disk is a real bonus. I try to run a paperless office, and scan every document that crosses my desk. Since I generally shred the documents after, it's critical that I don't lose the document images. I'm a software developer, so not losing my source code is critical, too. If I can keep all of these plus my entire photo library on Crashplan, that will be perfect. I will also back them up locally, but I need that "guaranteed" backup that I know I can get in the event of any type of disaster. I'll have to play with TimeMachine when I get set up to see what works best. Sounds like I might just let it backup all data.

Now if only they could hury it up with the new iMac. I'm getting impatient

 

[flynz4]

Thanks alot.

Bandwidth isn't an issue with me, as I have unlimited. In fact, I just discovered that they're installing a new fibre optic network with vastly improved speeds (still unlimited). Top speeds are 30Mbps upload and 200Mbps download, which will be very nice.

I'm going to switch to Crashplan when my Carbonite plan expires. I like everything I read about it. The apparently do not throttle, like Carbonite does. The seed disk is a real bonus. I try to run a paperless office, and scan every document that crosses my desk. Since I generally shred the documents after, it's critical that I don't lose the document images. I'm a software developer, so not losing my source code is critical, too. If I can keep all of these plus my entire photo library on Crashplan, that will be perfect. I will also back them up locally, but I need that "guaranteed" backup that I know I can get in the event of any type of disaster. I'll have to play with TimeMachine when I get set up to see what works best. Sounds like I might just let it backup all data.

Now if only they could hury it up with the new iMac. I'm getting impatient

I assume that you are new to Mac? I'm not positive.

For a paperless office... the "standard" is a program called DevonThink with a Fujitsu ScanSnap S1500M. It is an incredible combination and highly recommended. It utilizes artificial intelligence and fuzzy logic... but as you start scanning and filing... it learns and will automatically suggest the right place to file new things. It's search capabilities are also unmatched. As I scan our documents, DT performs OCR and saves every document as PDF+TEXT.

For your photo library, you should either use Lightroom or Aperture 3. A3 is Apple only, LR is cross platform. I own both... but you typically would not use both. They both outperform each other in various areas. On the whole, I like A3 better because if its incredible flexible workflow. In the end however, it is probably a coin toss. I spend a lot of time in my photo library now... and I cannot believe that I used to manually file things away in windows folders. A3 is just so much more powerful. It is the difference between being able to "store" your pictures and "use" your pictures. Night and day difference.

Like you, I want a new iMac. My wife and I have been sharing one iMac, and we each have our own MBAs, iPads, and iPhones. I really want my own dedicated iMac now... and I am looking forward to connecting my 27" ATD to it to give me dual 2560X1440 monitors. Some programs (like A3) are designed to really take advantage of dual displays.

/Jim

 

[Lastmboy]

Yes, I'm not even a Mac owner yet. I own an iPad, iPhone, and iPod, but not a Mac. I love my devices, and after playing with an iMac in store for a couple of hours, I kind of fell in love with it. I love how everything is so intuitive, and the work flows seem so natural. Mobility isn't a big deal with me, and my iPad (3) handles what I need for that. I like BIG, so I'll definitely be going dual 27" display. I was even wondering about picking up a refurb iMac to use as the second display (and to have something to work with right away), but not knowing what the new iMac will be kind of leaves some confusion.

Your scanning setup sounds perfect for me. I have a commercial high speed scanner that I absolutely love (Canon DR-2580c). Throw a stack of papers in, push a button, and they're all sitting in PDF/text searchable files in a folder waiting to be filed. However, having software that assists with filing/sorting/searching would really be slick. The sad part about my scanner is that Canon doesn't make OSX drivers for it. Not sure if I can make it work with something like Parallels or not.

For photos, I have always used ACDSee Pro (which does have an OSX version). However, I have been very intrigued with Aperture. ACDSee has lots of nice features, but navigation is really cumbersome. I've only played with Aperture in the store, but it really seems to "flow" nicely, as do most of the Apple apps. I started imagining the potential for my photos when they're not just sitting in a few hundred folders on my pc

From my very little experience on a Mac, it seems to be a different way of thinking, but if I don't get stuck in my old ways, the approach seems to actually make more sense. I've gotten so used to my iPad, that the gesture concept on the Mac is great.

 

[flynz4]

Yes, I'm not even a Mac owner yet. I own an iPad, iPhone, and iPod, but not a Mac. I love my devices, and after playing with an iMac in store for a couple of hours, I kind of fell in love with it. I love how everything is so intuitive, and the work flows seem so natural. Mobility isn't a big deal with me, and my iPad (3) handles what I need for that. I like BIG, so I'll definitely be going dual 27" display. I was even wondering about picking up a refurb iMac to use as the second display (and to have something to work with right away), but not knowing what the new iMac will be kind of leaves some confusion.

Your scanning setup sounds perfect for me. I have a commercial high speed scanner that I absolutely love (Canon DR-2580c). Throw a stack of papers in, push a button, and they're all sitting in PDF/text searchable files in a folder waiting to be filed. However, having software that assists with filing/sorting/searching would really be slick. The sad part about my scanner is that Canon doesn't make OSX drivers for it. Not sure if I can make it work with something like Parallels or not.

For photos, I have always used ACDSee Pro (which does have an OSX version). However, I have been very intrigued with Aperture. ACDSee has lots of nice features, but navigation is really cumbersome. I've only played with Aperture in the store, but it really seems to "flow" nicely, as do most of the Apple apps. I started imagining the potential for my photos when they're not just sitting in a few hundred folders on my pc

From my very little experience on a Mac, it seems to be a different way of thinking, but if I don't get stuck in my old ways, the approach seems to actually make more sense. I've gotten so used to my iPad, that the gesture concept on the Mac is great.

When I was a PC user, I used ACDSee as well... but mostly because it was a lightning fast photo viewer. I still organized and stored my pictures in Windows folders. If you want to get a good overview of the capabilities of A3, buy Robert Boyer's ebooks. I think it is photo.rwboyer.com and contains a wealth of knowledge presented in a no holds barred, very (good) opinionated manner. I also follow is blog postings on that site. Two of his ebooks (organization and file management) should be required reading. I often email him when I am confused and he is extremely helpful. His ebooks are probably the best deal in digital photography. Be forewarned... if you read his books, you will almost definitely end up buying A3.

/Jim

 

[Lastmboy]

When I was a PC user, I used ACDSee as well... but mostly because it was a lightning fast photo viewer. I still organized and stored my pictures in Windows folders. If you want to get a good overview of the capabilities of A3, buy Robert Boyer's ebooks. I think it is photo.rwboyer.com and contains a wealth of knowledge presented in a no holds barred, very (good) opinionated manner. I also follow is blog postings on that site. Two of his ebooks (organization and file management) should be required reading. I often email him when I am confused and he is extremely helpful. His ebooks are probably the best deal in digital photography. Be forewarned... if you read his books, you will almost definitely end up buying A3.
/Jim

Thanks. I'll check out the ebooks.