Time Machine to NAS to Cloud [101]

[zettabyte]

Hello guys,

I've searched a bunch on this and have found a lot of the posts to be older with minimal info. so here we go...

***Update: 07/14/15***

My Set Up:

[rMBP w/ 1TB OWC SSD], connected to =>

(1st Line of Back Up) - LOCAL

=> [WD My Cloud DL4100 24TB NAS w/ 6GB RAM], where my TIME MACHINE Back Ups are stored on.


(2nd Line of Back Up) - LOCAL

=> [WD My Book 6TB HD], plugged into the back of the NAS via USB 3.0. This HD stores my important libraries.

(3rd Line of Back Up) - OFF SITE

=> [Crash Plan], which backs up my 1TB SSD on the rMBP. However...

Being a photographer I have more photos than a kid saying the word "No," before they reach 18, so storing these photos on my rMBP wouldn't be practical. Therefore, I have all my Adobe Lightroom photos stored on an External HD. This precious folder is called, "Photos."

Here's how it looks like:

I launch Adobe Lightroom and all the photos/videos are 'HOME," on my External HD #1. =>

(1st Line of Back Up) - LOCAL

=> I make a copy of the "Photos," folder to another External HD #2 that is specifically dedicated for just this one Folder.

(2nd Line of Back Up) - LOCAL

=> I then copy over the "Photos," folder to my NAS #3, overnight.

(3rd Line of Back Up) - OFF SITE

=> I upload the "Photos," folder to my Amazon Cloud.

(4th Line of Back Up) - OFF SITE

=> I have the "Photos," set to upload to Crashplan.

(5th Line of Back Up) - LOCAL

=> I burn all the completed albums (Weddings, Birthdays, Sporting Events...) in segments to a DVD and keep that spool in my fireproof Safe.

(Cherry on Top ~ Insurance policy) - OFF OFF SITE

I will be swapping with a buddy to use his NAS in exchange for mine so that I'll have 1 on site + 1 off site (NAS) + 1 off site (NAS MIRROR, 5TB HD plugged into the back of the NAS, USB 3.0 slot)

***

Some of these questions may be rhetorical.

1. Are there any other ideas that I can implement to make it better/efficient/more thorough?

2. Can you partition the NAS drive to set the limit for Time Machine Back Ups? [YES. It's recommended to let the NAS complete a back up first, before adjusting the limit. It is also recommended to allot this space to double your HD capcity. ie..if you have a 1TB HD, then set the TM limit to 2TB (at minimum).]

3. What is the best file system partition to use for the NAS drive? HFS+ Journal? [It doesn't matter the NAS will work with it all.]

4. For Security Junkies like myself would it be possible to create a 256AES sparebundle within the NAS volume that Time Machine is using? This would mean that the image would always have to stay mounted yes? [NO, the NAS does not play nicely with this as you won't get a prompt to unlock the encrypted sparsebundle to even open it.]

5. When manually uploading to a Cloud, (because I don't trust them) is it best to upload large encrypted sparsebundles? What size limit should I keep the sparsebundle images to? 4GB? So that I could burn them to a DVD if needed? [YES this works. YES it's a good idea to keep the sparsebundles to 4GB so that you can easily copy them over to DVD's.]

6. What Cloud base back up service would you recommend between Crashplan, BackBlaze and Amazon? I've read a ton on these and Crashplan is looking really good....hmmm.. [You can't go wrong with either one of them. Crashplan is the one I went with because it does not delete files that it doesn't see after 30 days. ie....Backblaze will delete any backed-up files if it doesn't see it for more than 30 days. I've confirmed this with my colleagues who use Backblaze. Because I use multiple hard drives and store older photos, this would mean that my back ups of the older photos would be deleted off of Backblazes' server after 30 days of not seeing those files. Hence, why I went with Crashplan...I also have Amazon too for more of a manually drag+drop back up.]

7. As a photographer I have tons of photos and use Adobe Lightroom...is it best to create an encrypted sparebundle, place my Lightroom file inside it, then upload that to my server? Sparsebundle's forever expand yes? When replacing the file, since it is in bundles it will only replace the new or changed files instead of the entire file like a regular image would....[YES, this is a good idea and using this method works very well since only the changed bundles are being uploaded.]


Much appreciated guys...and I hope this is helpful for future readers as well...hence, the rhetorical questions....

 

[ColdCase]

My head is spinning..... Are you saying you are backing up the time machine files on your local TimeMachine destination to a NAS device (i.e. backing up a backup?).

Many photographers use flickr or something like it for off site backup. But that may not be secure enough for you.

I have my off site server set up with an encrypted link to an encrypted storage volume, and just do file transfers.

If you have high security anxiety, no remote or offsite storage that you don't own or control is safe. For my clients with proprietary data, I VPN into their server farm. All the apps and data is run on their secure server. Even that is not good enough protection against a persistent adversary

 

[zettabyte]

My head is spinning..... Are you saying you are backing up the time machine files on your local TimeMachine destination to a NAS device (i.e. backing up a backup?).

Many photographers use flickr or something like it for off site backup. But that may not be secure enough for you.

I have my off site server set up with an encrypted link to an encrypted storage volume, and just do file transfers.

If you have high security anxiety, no remote or offsite storage that you don't own or control is safe. For my clients with proprietary data, I VPN into their server farm. All the apps and data is run on their secure server. Even that is not good enough protection against a persistent adversary

Haha, sorry for the confusion...

Here are my goals:

1. To MIRROR my rMBP Hard Drive, which TIME MACHINE will do. (On Site) + Have my entire rMBP Backed Up off site with: for example, Crashplan.

2. To diversity my Adobe Lightroom master file multiple times on site and off site. (NAS w/ another HD connected to it as a MIRROR + Cloud). I would more likely create sparebundle files and archive the pictures, then upload that sparsebundle to the Cloud. This would obviously have to be done manually, which I don't mind too much.

3. Can you back up your NAS to Cloud services like Crashplan??

As for security, I use a VPN religiously and have all my passwords 50 characters. I got the feeling you didn't feel like encrypting a sparebundle with AES256 with a 50 character password is sufficient enough to upload to my server?

For example: I'd create a sparebundle for one of my wedding picture archives (AES256+50CharPW) then that sparsebundle would be uploaded to Crashplan, yes? No? Isn't that one advantage of sparsebundles in that it would only upload any changes to the bundle? Like how Time Machine works....

Regarding my #4 Question in the OP, I was just curious if it'd be ideal to create a 2TB sparsebundle on one of my external HD's (2TB) as a layer of security. I've actually done this to my 256GB USB 3.0 Flash and it works great...it may be crazy to some, but I think it's pretty awesome LOL....It's funny seeing a mount within a mount in my Finder haha...

Anyhow...thanks for the reply. I'm looking forward to your input.

Cheers..

ps. Nice machine rMBP's rock.

 

[ColdCase]

OK I see what you are asking and you seem to know what you are doing. I don't think I can help. The advantage of a sparse bundle over a fixed storage volume size is that it will dynamically resize/grow (important perhaps when multiple machines share the same drive/volume). Whether its spare or fixed, only the files that are changed are saved during each backup cycle. The disadvantage of sparse bundles is that more things can go wrong. I typically don't use sparse bundles, just fixed size encrypted and access controlled volumes. I don't use crashplan because its performance is less than stellar in my area, can't handle the larger GB video files I work with. Can you put a mountable drive on crashplan? I dunno.

Similar to you, I set up time machine to backup the OS and boot drives to two different disks, one directly attached, one on a MacOS server. I use CCC to do the same and also backup my working and archive libraries with CCC. Off site, currently its a manual thing.

My security comments was more in line of what kind of threat you are trying to protect yourself against, how valuable is the data and how persistent and motivated the adversary is going to be. The weak point may not be in the encryption, but in key management. So its a judgement call of whats good enough, and what you are doing is pretty good.

Anyway, sorry I'm little help, but I'm not sure sparse bundles buy you any more than fixed size volumes, but I probably missed something. I think CCC supports what you want to do and automatically. It does sparse bundles and encryption, but I dunno if crashplan supports the concept. Don't they have their own app?. CCC has knowledge base articles that may provide some useful info: http://bombich.com/kb/ccc4/search/Backing up to a disk image You may already be familiar with the contents, and I don't think they mention crashplan.

You may have to wait a week, but someone else with more direct knowledge should be jumping in here.

 

[bgd]

2. Can you partition the NAS drive to set the limit for Time Machine Back Ups?

I am doing this on a Synology NAS so it's possible. Can't comment on your WD though.

6. What Cloud base back up service would you recommend between Crashplan, BackBlaze and Amazon? I've read a ton on these and Crashplan is looking really good....hmmm..

I'm using CrashPlan and it seems to work. Never had to restore in anger, mind. I did a reasonable amount of research at the time and CrashPlan seemed to suit me best. Can't remember the details now. Probably mainly feedback from users on this site.

3. Can you back up your NAS to Cloud services like Crashplan??

Yes, I'm currently doing this but via a MBA which is connected to the NAS shares. It only works if the share is visible so you need to monitor it. I think the plan I have allows me to back up 1 computer to the Cloud. That obviously includes any drives visible to that computer.

Have a look at www.hanselman.com if you want to do it directly from the NAS. He is talking about Synology though.

I'm actually changing my set up. I now use a mini as a server with DAS and have migrated all my storage from the NAS to the DAS. I will change CrashPlan to backup the DAS and the NAS will become local backup.

 

[TechpumpkinWD]

Hi there,

I think I can help with the WD My Cloud questions. In all of WD My Clouds you cannot limit a share, but you can limit the storage space that each user uses. For example, if you want to limit the space your Time Machine Backup will take to 200 GB. You must go to the TimeMachineBackup share and change it from public to private, then setup a User that you'll use only for Time Machine and limit the space that this User can use to 200 GB.

As for the NAS file system – you cannot change the file system and there isn't really any need to do so, as the NAS itself is doing all the reading/writing on the HDDs. It doesn't matter if you are accessing it from a OS X, Linux or Windows. It uses ex4t. Hope you find this information useful.

 

[zettabyte]

Thanks guys for all the great info. and input.

Here is my set up at the moment:

rMBP --> Time Machined to --> WD My Cloud 4TB (I've partitioned 1TB for TM)

As for my Adobe Lightroom photo files: I've been manually just dragging and dropping the "photo," folder that contains all the 40,000+ photos to my NAS and also an external HD so there are two back ups.

Q: What is the absolute best CCC like program where I'd be able to set up a back up schedule that also does a thorough verification of the copying? I know the drag and drop method leaves a lot of room for error/corruption of files due to the amount of data that I am copying over manually, since there is no verification of any kind. I've seen the program "Retrospect," but it seems pretty pricey...I have a hunch that there is a more affordable option .... Is CCC a good option to do the trick?

Now I just need to get CrashPlan (as my off site back up) as that seems to be what fits the bill for everything that I am needing. However, one thing that I like about BackBlaze is that you can control individual folders for back up, which would be very handy for my Lightroom files...Is this possible with CrashPlan as well?

Q: Would it be best to Time Machine to another External HD connected via USB as a secondary back up for TM? -OR- Should I just back up the TM file that is saved on the NAS manually to an external HD?


NAS Set Up Questions: (A lot of this is very rhetorical on purpose...ie...thinking out loud so that you can see what's turning in my head so that I can get a second opinion in case that I'm missing something...)

1. Is it best to have a stand alone NAS (ie...WD My Cloud 4TB) and hook up an external 4TB HD to the back of the NAS via USB 3.0 to use for Safepoint back ups? -OR- To just buy an 8TB 2-Bay My Cloud that has one 4TB mirroring the other 4TB HD inside the one unit? Isn't both methods essentially the same in terms of having two back ups at the end of the day, except that the 2-Bay would be more seamless, since you don't have to set up the Safepoints as an additional step?

(Not that scheduling Safepoint is hard to set up, the point here is that since Safepoints are on a schedule, there is a time gap where you wouldn't have a direct mirror at all times depending on how often you set it up to perform the Safepoint back ups. Having a 12 hour gap for when Safepoint does its thing can be detrimental for photographers as you can imagine in the "likely" event somewhere in the future when the NAS decides to kick the bucket.)

The 2-Bay with the RAID 1 / Mirror seems more fluid since it is constantly being mirrored 1:1, and I like the fact that if one drive fails you can just open the bay and replace the fallen drive. However, it is a bit more pricey where an 8TB, which is essentially a 4TBx2 is just 4TB.


2-Bay Set Up:

WD My Cloud 2-Bay 8TB Mirror (4TBx2) is around $450.

....which is the same (only storage wise) as buying....

WD My Cloud 4TB $195 + WD 4TB HD $135 = Roughly $330

So with a difference of about $120, I guess the decision is whether;

1. I want the fluidity of having the 2-Bay 4TBx2 'RAID 1' set up with the 1:1 at a cost of $450.

2. I can pay about the same amount ($450) to buy a standalone 6TB WD My Cloud + 6TB WD External Hard drive to have an additional 2TB of storage space over the 4TB with the 2-Bay RAID 1 set up.


Summary (Entire System):

Back Up 1 => rMBP Time Machined to my NAS.

Back Up 2 => rMBP Time Machined to an External HD plugged directly into the computer.

Back Up 3 => CrashPlan. (Off Site)

Note: I don't have all my Adobe Lightroom photos on the rMBP's hard drive, instead I have all the photos hosted on:

Primary / 1st Copy => ...a dedicated External HD #1 that is only for Lightroom photos.

Back Up 2 => ...I then back up this "photo" folder from HD #1 that contains everything to another External HD #2.

Back Up 3 => ...I also make a third copy of this "photo" folder to my NAS.

Therefore, unless I am able to select certain folders individually with CrashPlan, it would only be my rMBP that is being backed up, otherwise I'd have to put all these photos to my main SSD on the computer, which isn't going to happen. Hence, why I have to use an External HD as a Home for my photos.

The pros to this is that the back up to CrashPlan would be must faster/smoother (only my rMBP files) since my photos which take up the majority of the space is on an External...so I guess to keep these two separate can be a good thing.

In this case, would Crashplan be the best option over BackBlaze, Amazon etc...where I can be in control of what is being backed up? I have yet to see the full interface of Crashplan to know if I'd be able to manually select the 'photo' folder that's on an External HD to back up...


Thanks guys!

 

[zettabyte]

***[Update: 07/03/15]***

I've ordered the WD DL4100 24TB NAS. Don't even bother with the third Option (EX2).

Ok guys...Decisions, decisions, decisions....I'm going to upgrade, but not sure what to yet...

Here are my prospects:

Option 1: WD My Cloud EX4100 (4-Bay) - 24TB Configuration
- 1.6GHz Dual Core
- 2GB RAM (Not Expandable)

-OR-

Option 2: WD My Cloud DL4100 (4-Bay) - 24TB Configuration
- Intel 1.7Ghz Dual Core (Atom C2338)
- 2GB RAM Stock (Expandable to 6GB)

Although the DL Series (Option #2) is newer and better all around, it is about the same price as the older model. Both units run a little under $1,500 at the moment. I'd configure these to probably RAID 5, which would leave me with >12TB of useable space.

-OR-

Option 3: WD My Cloud EX2 (2-Bay and is just a RAID 1 Mirror) - 12TB Configuration
- Nothing fancy, just two drives with one primary and one mirrored back up. (What I'd use it as: RAID 1)
- This would leave me with a hair under 6TB of usable space.

This maxed out unit with 12TB is around $650.

Does anyone have any experience with these NAS units? I'm curious if there's a noticeable difference in real world speed between Option #1 and #2? They both come with the same amount of RAM and the CPU's are relatively identical. There's a reason why Option #1 is even on the list of being considered, though it is an older model. As for Option #3, I'm just gun shy of whether it'll give me the extremely slow performance as my current one does now. The reason it's on the list is in the event it is faster....anyone? Is it?

I am upgrading because the regular 4TB MyCloud NAS (the bottom of the food chain model that has a white case) that I have is extremely slow. It's almost worthless and not practical for mobile devices or any remote access. I've tried everything and no it's not my internet, hardware, ISP or router as they are all top of the line. These base units just plain out suck and not worth the money.

Any input would be greatly appreciated.

 

[bgd]

There does come a time when you have to bite the bullet and buy something. Don't think you will go wrong with that.

Going back to your earlier post. As long as the folder is visible to CrashPlan then you can back it up. All my files are on an external drive and they are backed up to CP. You can be quite specific about the folders you specify.

CP is quite versatile. In addition to the offsite back up you can use it to back up locally. I haven't tried that yet but basically it looks like a one stop shop for all backup needs.

And just a note on Lightroom. Make sure, in addition to the photo files, your catelogue is being backed up too. Mine is covered by TimeMachine, but I also take an irregular backup and put it on my server so that CP will pick it up.

 

[zettabyte]

There does come a time when you have to bite the bullet and buy something. Don't think you will go wrong with that.

Going back to your earlier post. As long as the folder is visible to CrashPlan then you can back it up. All my files are on an external drive and they are backed up to CP. You can be quite specific about the folders you specify.

CP is quite versatile. In addition to the offsite back up you can use it to back up locally. I haven't tried that yet but basically it looks like a one stop shop for all backup needs.

And just a note on Lightroom. Make sure, in addition to the photo files, your catelogue is being backed up too. Mine is covered by TimeMachine, but I also take an irregular backup and put it on my server so that CP will pick it up.

Thanks for the confirmation with CrashPlan. After careful evaluation on all the available Cloud choices, CrashPlan really tops them all in many respects. It's by far (arguably if one is looking at it from the outside, but inarguably if one understands the in's and out's thoroughly) CrashPlan is the best choice.

As for Lightroom, yes I agree. Regardless of any auto back ups of any kind, I always back up the catalog and settings manually.

Great stuff....

Cheers!