Best network backup solution?

[eogold]

I need a solid, reasonable fast network backup solution for two MacBook Pros. A business depends on these machines and thus a very robust system is required (zero tolerance for data loss). Here's what my needs are:

-- Network backup for two new MacBook Pros, over a wireless network
-- Latest Apple Extreme Base Station
-- A RAID 1 device with hot-swappable 3.5" SATA drives

My plan is to buy three 2 TB 3.5" SATA drives and rotate one of the drives off-site on a weekly basis for added security, and to make using an online backup service like Mozy (for offsite redundancy) unnecessary.

I welcome any suggestions, especially specific brands or hardware configurations.

Thanks!

Eric

 

[zhenya]

I would buy a backup device that is compatible with Time Machine and use that. It works as well or better than even the very expensive corporate backup software i've administered.

I'm not clear how the RAID 1 setup you propose would work. Re-mirroring a raid volume takes a long long time, and even if you can do it hot, performance will be severely degraded while it is working. We're talking like 24 hour plus rebuild times with that much data. It's not really how Raid is meant to work.

I would back up to the network drive and then to the cloud as my off-site backup. It's more reliable, more secure, and less of a hassle because there is no user input needed. I'm not sure why you think NOT doing this would be a benefit. The only reason I can see not doing it is in case of a very slow network connection and/or many gigabytes of changed files every day.

 

[eogold]

RE: Best network backup solution?

I would buy a backup device that is compatible with Time Machine and use that. It works as well or better than even the very expensive corporate backup software i've administered.

I'm not clear how the RAID 1 setup you propose would work. Re-mirroring a raid volume takes a long long time, and even if you can do it hot, performance will be severely degraded while it is working. We're talking like 24 hour plus rebuild times with that much data. It's not really how Raid is meant to work.

I would back up to the network drive and then to the cloud as my off-site backup. It's more reliable, more secure, and less of a hassle because there is no user input needed. I'm not sure why you think NOT doing this would be a benefit. The only reason I can see not doing it is in case of a very slow network connection and/or many gigabytes of changed files every day.

I KNOW the above setup works because I have sen it in action with my own eyes.

When you swap in the "third" drive, the RAID hardware immediately begins cloning the new drive from the old, and for 500-1000GB drives it does take some time. But that's no big deal, you just let the re-mirrowing run overnight, and what you end up with is very fast over-the-network backups (and if you have large Photoshop or multimedia files that you deal with, backups over the Internet to online services are exceedingly slow, even on relatively fast internet connections). Plus, if a catastrophic event occurs (house floods, burns down, gets robbed, massive power surge, etc.) you simply retrieve your off-site backup drive and restore from that immediately (and if it's a Time Machine or SuperDuper backup, your business is back and running in only a few hours), as opposed to the hassle of retrieving your data files from your online backup service (which usually are just your data files and don't include all your system files, applications, etc.)

My understanding is this is EXACTLY the way RAID 1 is meant to work.

 

[zhenya]

Yes, it does work that way, but what I'm saying is that it's not mean to be rebuilt like that on a weekly basis. I would test out how long the rebuild takes and what the performance is like during the rebuild. I don't think it is going to just take overnight with that much data. I've seen 160gb drives take upwards of 30 hours to rebuild. It's just not a method I would trust my data with - especially because you now are involving all three copies of your backup in the process every week. If something goes wrong you run the risk of corrupting all three at once.

Some cloud backup services like crash plan will allow you to seed your backup to them via a drive they send you if you have a very large amount of data. In case you need your data back, they will ship it to you overnight on a drive. The data is encrypted and completely inaccessible by anyone without the cryptographic key. Data is then backed up at the block level, so only the changes to the files are sent, not the entire file unless it is new. This makes it very efficient. You can back up any and all data to the cloud, just like a local backup.

I understand that this may still not work for you, but I'd suggest you modify your off-site plan to be a rotation of external esata drives rather than involving the raid set directly.

 

[ibennetch]

I KNOW the above setup works because I have sen it in action with my own eyes.

I've seen a lot of things "work" that are bad ideas. I'd put this in that category.

My understanding is this is EXACTLY the way RAID 1 is meant to work.

RAID 1 is meant to keep the array running if a disk goes away -- not to swap in and out drives as a backup means. If that's your understanding of RAID, you should probably at least read a lot more about it before setting up your zero tolerance backup set...and if the data is really that critical, a consultant who specializes in this sort of thing might be a good idea and wise investment.

Just because what you propose will work doesn't make it a good idea. In your proposed usage, what happens the active drive goes bad in the middle of one of the overnight refreshes? What if the RAID controller gets confused and mirrors the week-old drive over the fresh data (which shouldn't happen, but has)? You should consider the extra wear and tear you're putting on the primary drive, too; copying the entire contents of the drive once a week...remember that this system won't do a differential backup of the old drive, but will begin the backup from a blank slate every week. That means you're stressing the only good copy of your data at the time when it's most vulnerable.

Your system seems pretty good otherwise (though if you're serious about the "zero tolerance for data loss" part, like losing data will at the very least mean the end of your business and possibly ruin people's lives [I'm thinking of a bank, for instance], then I think you need a few more layers). I'd probably use RAID1 for the first step, like you planned, but without swapping the drives (depending how near-line this first data set needs to be, maybe not RAID at all). I'd then have several external drives with an rsync or rsync-like program to do the copy from that first stage backup to the first external drive which I'd rotate out daily, and probably another set of drives for a weekly backup. Again, this depends on how "near" you need to have your potential restores and how fresh you need the backups.

Then I'd test the heck out of every step to make sure you can restore from any potential situation.

I'm away from home at the moment so can't check, but think the book to get is O'Reilly's "Backup and Recovery".

 

[Blue Velvet]

I welcome any suggestions, especially specific brands

I wouldn't rely on Time Machine. Last time I set up a network backup on a server for our inhouse studio, I used Retrospect for incremental backups. Worked flawlessly for years with no problems at all and in one instance, was invaluable in recovering a lost week's work. After too many problems, I don't even use Time Machine at home, preferring SuperDuper instead.

 

[willieva]

How much data are you talking about? Wireless backups are slow.

 

[zhenya]

I wouldn't rely on Time Machine. Last time I set up a network backup on a server for our inhouse studio, I used Retrospect for incremental backups. Worked flawlessly for years with no problems at all and in one instance, was invaluable in recovering a lost week's work. After too many problems, I don't even use Time Machine at home, preferring SuperDuper instead.

Hmm, that's funny. We used Retrospect for years and while I thought it was a decent backup solution, I think Time Machine is every bit as good for a small backup of Mac only computers like this. Plus it takes virtually no configuration.

 

[talmy]

Since notebook computers tend to move about, I'd suggest using TimeMachine to a networked drive (so backups are basically continuous while the notebooks are on and in range of the network). And then cloning the networked drive, taken offsite on a regular basis.

I don't fully trust TimeMachine and do a weekly system clone as well. This can be done using Carbon Copy Cloner or SuperDuper!

There are certainly various levels of paranoia to determine frequency. (IMHO, paranoia is good when it comes to making backups). If it would be hard to frequently take backups offsite (at my recent employers, they do this daily) using a cloud-based backup of the network drive would be advised.

 

[eogold]

RE: Best network backup solution?

Here's what I've decided upon, with the following considered:

-- Wireless backup over the network of large files will be a bit slow
-- Noise from fans is a concern
-- Ability to easily take a full backup on the road
-- Online backups are slow and not convenient if a full recovery is necessary.

Two identical setups for the two users, as follows:

-- Quantity 1: NewerTech's Guardian MAXimus mini RAID (Raid 1, 2 x 500GB 2.5" drives, fanless)
http://eshop.macsales.com/owcpages/guardianmax_mini.html

This will connect via FW800 to the MBP. These drives will hold a mirrored TimeMachine backup of the MBP internal hard drive.

-- Quantity 2: OWC On-The-Go Pro, 500GB Triple Interface (500GB).
http://eshop.macsales.com/item/Other World Computing/MS8U5500GB8/

These will be configured to hold a bootable clone (using SuperDuper) of each MBP's internal hard drive. At any given time one of these drives will be connected to the MBP and the other will be offsite. These two drives will be rotated (offsite-to-onsite) on a weekly basis. When on the road, one of these drives will travel for on-the-road backup, and the other stay home, offsite.

Cost per MBP: $470

Unless I've missed something, the above plan, while a bit pricy, ensures fast, redundant "duplicate" and "versioned" backups, protection against on-site catastrophe/theft, minimal full recovery time in the event of MBP hard drive failure, full backup coverage while on the road.

Thoughts?

Thanks,
Eric

 

[radek42]

Here's what I've decided upon, with the following considered:

-- Wireless backup over the network of large files will be a bit slow
-- Noise from fans is a concern
-- Ability to easily take a full backup on the road
-- Online backups are slow and not convenient if a full recovery is necessary.

Two identical setups for the two users, as follows:

-- Quantity 1: NewerTech's Guardian MAXimus mini RAID (Raid 1, 2 x 500GB 2.5" drives, fanless)
http://eshop.macsales.com/owcpages/guardianmax_mini.html

This will connect via FW800 to the MBP. These drives will hold a mirrored TimeMachine backup of the MBP internal hard drive.

-- Quantity 2: OWC On-The-Go Pro, 500GB Triple Interface (500GB).
http://eshop.macsales.com/item/Other World Computing/MS8U5500GB8/

These will be configured to hold a bootable clone (using SuperDuper) of each MBP's internal hard drive. At any given time one of these drives will be connected to the MBP and the other will be offsite. These two drives will be rotated (offsite-to-onsite) on a weekly basis. When on the road, one of these drives will travel for on-the-road backup, and the other stay home, offsite.

Cost per MBP: $470

Unless I've missed something, the above plan, while a bit pricy, ensures fast, redundant "duplicate" and "versioned" backups, protection against on-site catastrophe/theft, minimal full recovery time in the event of MBP hard drive failure, full backup coverage while on the road.

Thoughts?

Thanks,
Eric

Hi,

I was looking into RAID1 back up solution for home. I considered different network attached storage (NAS) similar to ones you mentioned. At the end, I decided against that since each RAID controller uses different algorithm to distribute data. Everything should be fine if one of the hard drive dies, but it seemed to me that if controller dies your disks might be useless (unless you can find exactly same controller). If I am wrong please let me know.

At the end I went for simple linux box with two hard drives; one for primary back up (over wireless network using incremental back up with rsync from linux and windows machines) and that syncing the first hard drive with the second one every night. It's not perfect, but it serves me well so far.

For additional/offsite back up I use WD Book to sync every now and then.

Hope this helps.

R>

 

[eogold]

Hi,

I was looking into RAID1 back up solution for home. I considered different network attached storage (NAS) similar to ones you mentioned. At the end, I decided against that since each RAID controller uses different algorithm to distribute data. Everything should be fine if one of the hard drive dies, but it seemed to me that if controller dies your disks might be useless (unless you can find exactly same controller). If I am wrong please let me know.

At the end I went for simple linux box with two hard drives; one for primary back up (over wireless network using incremental back up with rsync from linux and windows machines) and that syncing the first hard drive with the second one every night. It's not perfect, but it serves me well so far.

For additional/offsite back up I use WD Book to sync every now and then.

Hope this helps.

R>

I'm not sure what happens "if the controller dies", but since the scheme above also includes a separate bootable backup to an external FW drive (via SuperDuper), you're covered.

A zillion folks everyday use RAID 1 for mirrored backup of critical data, so if the problem you describe was at all common, I assumed we'd all hear about it.

 

[zhenya]

That sounds pretty good Eric. May I ask why you are giving up on the single NAS? After the initial backup (which could be done via ethernet - gigabit is faster than fw800), wireless should be fine. In general i've found over the years that you want to make backups as automatic as possible, as any user input is a failure point. Remember, that if these folks take their laptops home at night, they will have to plug in these drives every day, year after year. An even bigger improvement would be if the offsite drives could be plugged directly into the NAS and cloned automatically via a script.

Don't worry about the raid controller - raid 1 drives are bootable independently. It's other raid types that require the same controller they were built on to boot.

 

[eogold]

That sounds pretty good Eric. May I ask why you are giving up on the single NAS? After the initial backup (which could be done via ethernet - gigabit is faster than fw800), wireless should be fine. In general i've found over the years that you want to make backups as automatic as possible, as any user input is a failure point. Remember, that if these folks take their laptops home at night, they will have to plug in these drives every day, year after year. An even bigger improvement would be if the offsite drives could be plugged directly into the NAS and cloned automatically via a script.

Don't worry about the raid controller - raid 1 drives are bootable independently. It's other raid types that require the same controller they were built on to boot.

My concern about wireless backups is that they would be a bit slow for these particular clients to tolerate. They both work a lot with fairly large audio files (typically like podcasts). Questions:

-- Does anyone have any real world experience with Time Machine backups of fairly large files over an 802.11n-connected NAS?

-- How long would an initial Time Machine backup of about 350 GBs of data take over wired gigabit ethernet?

-- If I do go with a NAS -- any suggestions for brands/models?

Thanks, Eric

 

[talmy]

I've had no problems with wireless TimeMachine backups. Remember it only has to copy files that have been created or modified in the past hour. The initial backup can take a long time (many hours) so is really best done wired.

 

[annk]

I wouldn't rely on Time Machine. Last time I set up a network backup on a server for our inhouse studio, I used Retrospect for incremental backups. Worked flawlessly for years with no problems at all and in one instance, was invaluable in recovering a lost week's work. After too many problems, I don't even use Time Machine at home, preferring SuperDuper instead.

This intrigues me, because I'm considering giving up Time Machine (I've been using the utility on an external HD, not on a TM unit from Apple) in favor of a different solution I won't go into here, so as not to derail or hijack the thread.

What sort of problems did you have with TM, and why did you finally decide to ditch it?

 

[Blue Velvet]

What sort of problems did you have with TM, and why did you finally decide to ditch it?

First off, I should say that for stuff at home and that includes freelance work, I rarely, if never care about incremental backups so straight out cloning is fine for me. At home, I've never had to retrieve a file I've accidentally trashed or wanted to roll back to an earlier version of something, keeping multiple versions of any design work as I go along and then clearing up the project folders before archiving. I also don't backup over a wireless network and I only back up one Mac... so all that may make my situation different from yours.

Tried Time Machine for a year or so with a firewire drive, it would only backup intermittently and then, on occasion, it would never backup at all, getting stuck into one of its loops where it would just be spinning its wheels for hours on end. Most importantly though, if my main drive stopped working, I'd have to get a new drive as you can't boot from a Time Machine backup and then face many hours of restoring from the TM machine backup, which, if it happened while I was trying to meet a deadline, would make life difficult.

As it happens, my main drive stopped working last week, but I was running off the cloned backup within a couple of minutes, all apps working, everything identical with only slight loss of data, a couple of emails and some notes. Basically, the way I see it, is that Time Machine puts all its ease of use and convenience at the backup stage, not the restoration stage. I prefer to have it the other way around, knowing that when the crunch comes, I'd rather pick up more or less immediately where I left off... which is what happened a few days ago.

Apart from the flakiness with some firewire drives, to me, Time Machine backups almost fall under the category of proprietary backups which Mudbug kinda warns against. Easy to grab some files from, but to fully restore back so that you're up and running within seconds? Not so much.

 

[annk]

Thanks, the business of ease-of-backup vs ease-of-restore is what I've been thinking about lately.

 

[zhenya]

First off, I should say that for stuff at home and that includes freelance work, I rarely, if never care about incremental backups so straight out cloning is fine for me. At home, I've never had to retrieve a file I've accidentally trashed or wanted to roll back to an earlier version of something, keeping multiple versions of any design work as I go along and then clearing up the project folders before archiving. I also don't backup over a wireless network and I only back up one Mac... so all that may make my situation different from yours.

I think that people looking for a backup solution should take this advice with a huge dose of caution. That's great that you've never had to restore a previous version of a file from backup, but I can tell you, from firsthand admin experience, that it is by far the most common reason someone needs to restore from backup. Straight 'copy' or 'clone' backups sound easier on the surface, and they are certainly much better than no backups, but it's no method I would ever recommend just because someone tells me they are wary of proprietary backups. A backup isn't a backup without testing that you can restore from it - so make that a part of your routine until you are comfortable with it.

My recommendation is that you do both a full-disk clone (once a year or so is ok) AND daily incremental backups. This way you get the benefits of quick restore to a working system, and the security of daily, automated, incremental backups. A backup routine should be automated (minimal human intervention) and tested regularly. Without both of those pieces, you are asking for data loss when you need it most.

Mudbug's warning linked to above was a result of poor planning, and not properly testing his backups. Nothing more, nothing less.

 

[Blue Velvet]

I can tell you, from firsthand admin experience, that it is by far the most common reason someone needs to restore from backup. Straight 'copy' or 'clone' backups sound easier on the surface, and they are certainly much better than no backups, but it's no method I would ever recommend just because someone tells me they are wary of proprietary backups.

Horses for courses. Whatever works for people and Time Machine doesn't cut it for me. The way I organise my work and essential files at home on my freelance setup is rigid, yet contains redundancy. Simply put, I don't trash files unless I've triple-checked what they are. In the studios I've managed from the technical side, I've always insisted on incremental network backups for each machine to our server and internal automated clones, as I briefly mentioned upthread when talking about Retrospect... so thanks for the tips, but I'm good, thanks.