Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

How would you backup 420 GB (and growing) of home HD video+pics

D

desmotesta

macrumors member
Original poster
Nov 17, 2012
55
0
Hello all,

I am at a crossroads and am looking for advice from people (that means YOU) who are far more informed than me.

I have been saving/archiving home Movies (HD video since 2009) + family pictures since 2001, ALL that data now amounts to a little over 420 GB.

I have 2 X 500 GB Western Digital hard drives. Both are archived with the exact data and serve as a secondary back up for each other.
One of the HD is connected to a PS3 so I can have instant access to all videos and pictures (on TV)- The 2nd HD serves as a back up OF the back up. (yes I am paranoid).

Since I am quickly reaching the 500GB mark ( and have a 3rd baby arriving any day) I need to purchase bigger HDs as soon as possible.

I see my future needs as follows:
- ability to connect 1 dedicated external hard drive to PS3/PS4 allowing quick access to all video/Picture archive on T.V

- Ability to connect the external hard drive to my MBP via Firewire to thunderbolt (or thunderbolt to USB 3.0, i hear it coming soon) and periodically download and archive all HD video/Pictures

- Each HD at 2 TB or more

Here is my plan, please let me know if it makes sense or if I should buy different/better hard ware - please feel free to recommend a different solution for my needs:

Purchase 2 X External hard drives, at least 2 TB or more, format one for Mac and the other for PS3 - Both Hard drives mirror each other and serve as a back up OF the back up. (using 2 hard drives to mirror each other since failure rate of optical HDs is high and High capacity SSDs are Very expensive)

Questions:

- Should I buy HDs with firewire or put up with USB 2.0 for now and wait for a thunderbolt to USB 3.0 adaptor?
- Should I pay extra for thunderbolt capable hard drives?
- Should I invest and research a RAID option?
- Is there another reliable archive option other than listed above (have already looked into Online archive services, they are very slow and will take me months to upload 420 GB of data)

Thank you all for your help
 
All of my impromptu videos are or can easily be snipped to under a minute.
Photos are resized on upload into Picasa 2048 res.

With this criteria, Google gives unlimited off site space. Safe from fire, flood. Tornado, theft, etc.

I'd lose some quality in photos... But 2048 res is acceptable, and memories not lost.

Carbonite is also a common paid service.
 
Menel said:
All of my impromptu videos are or can easily be snipped to under a minute.
Photos are resized on upload into Picasa 2048 res.

With this criteria, Google gives unlimited off site space. Safe from fire, flood. Tornado, theft, etc.

I'd lose some quality in photos... But 2048 res is acceptable, and memories not lost.

Carbonite is also a common paid service.
Click to expand...

I spoke to carbonite, lady on the phone told me it will take 6 months or more given my upload speeds to upload all 420+ GB
 
Menel said:
Check out CrashPlan
Click to expand...

its not the service, its the OP upload speed. but, with crashplan, you can opt for them to send you a harddrive and you can do a local upload which will be a million times faster and send it back to them. of course, this will cost extra. It took me about 3 months to upload about 2.2TB of data, but I have a relatively fast upload speed.
 
Just a note that Crashplan can't accept files larger than ~6GB. Many of my raw home movie captures exceed 6GB.
 
desmotesta said:
Questions:

- Should I buy HDs with firewire or put up with USB 2.0 for now and wait for a thunderbolt to USB 3.0 adaptor?
- Should I pay extra for thunderbolt capable hard drives?
- Should I invest and research a RAID option?
- Is there another reliable archive option other than listed above (have already looked into Online archive services, they are very slow and will take me months to upload 420 GB of data)
Click to expand...

Thunderbolt doesn't make sense for video archives. Buy USB 3 storage. USB 3 storage devices are backward compatible with your USB 2 hosts. Note that reliable 3TB drives are now less than $100, 4TB less than $150, they may be a better value for you.

Raid is not a backup solution. Simply mirroring the drives as you have been doing is better. No need to get fancy

Crashplan and others are suited for OS and everyday backups and versioning, not backup of video. I pay about $10 a month for unlimited web based FTP storage. So I have all my raw home video (3TB) on one drive backed up daily by CCC to a second drive. I have finished (edited) video on another drive, video that could be reconstructed by projects backed up by time machine. I have FTP'd much of the video off site... which can be accessed by anyone with proper authentication (family members) via a web browser. As you know, that can take awhile. But you have to start somewhere. You may want to save the priceless stuff onto a third drive and store that off site.
 
ColdCase said:
Just a note that Crashplan can't accept files larger than ~6GB. Many of my raw home movie captures exceed 6GB.
Click to expand...

CrashPlan advertises all over their site that they have no file size limits.

You can seed a CrashPlan backup by sending them a drive. That takes care of the up front bandwidth issue as well.

I'd look into the NAS solution, but here's my rather paranoid setup..
-I keep the main copy of such files on a 4TB drive in my Mac Pro
-I have Time Machine running for all my local files
-I have a local CrashPlan archive that saves to a server
-I have a cloud CrashPlan archive

The Time Machine setup is really just there for ease of use.
 
Last edited:
brentsg said:
CrashPlan advertises all over their site that they have no file size limits.

You can seed a CrashPlan backup by sending them a drive. That takes care of the up front bandwidth issue as well.
Click to expand...

Advertising and marketing are much different than real world experience with those guys. I took a free trial. Started backing up my MP and things seemed to be going well. In the morning I found it was stalled ~5 Bps data rate. Crashplan support was no help. After a few days of futzing around I discovered the stall was happening during a ~8GB file transfer. File would upload just fine until the 6GB point and it would slow to a crawl (estimated time was a couple years). After a couple rounds of questions with CrashPlan tech support they admitted their server can't handle anything over a ~ 6GB file size very well and told me to go away, politely.

So you load all your video onto their drive (it is expensive BTW) and send it to them to prime the pump. What if you happen to create another 8GB video? You're hosed. You can't back it up and they don't have a way to merge it even if you mailed it to them on a USB stick. Worthless, I think, for this purpose.

They advertise but can't deliver... Crashplan uses a number of servers around the world, some working better than others, so I dunno if you would run across this limit elsewhere.

Bottom line, don't use crashplan to back up video. They advertise unlimited file size, but don't mention throttling (reminds me of some cell phone plans). It seems to work well in most locations to back up OS and many smaller files, but there are also many folks with problems. I don't think any of these providers want to deal with large files but, on the other hand, 95% of consumers do not have large files and thats the market they focus on.

On a side note, I can't seem to git rid of the crashplan app. I send it to the trash but then the trash can't be emptied. I can't seem to remove it short of reformatting the drive. Crashplan support doesn't know whats going on, but I hear they are pretty good otherwise..

A NAS is an expensive overkill for what the OP wants to do. A better alternative is to forget the NAS and just go to a media server approach, he would be happier... but would have the same backup dilemma, and two independent mirrored drives would also be the best solution.
 
Last edited:
I lost a drive and was able to recover files larger than 6GB just fine via CrashPlan. In fact the drive I lost had all my video and also iTunes content. I seeded with a drive though.

I'll drop a couple huge ones into a folder that is archived and report what happens.
 
Do you have another family member that would be willing to have you drop a small server into one of their rooms? I'm going to set this up w/ my parents and use bittorrent sync. This way, I'll have a copy of all my media safely backed up at another site (their house), and I won't need to pay a hosting fee or anything for them to have it there. Bittorrent sync can sync huge files.
 
ColdCase said:
For grins try a single 12 GB file.
Click to expand...

I'm doing one that's nearly 12GB. So far so good, uploading at about 5mb/s.

----------
spaceballl said:
Do you have another family member that would be willing to have you drop a small server into one of their rooms? I'm going to set this up w/ my parents and use bittorrent sync. This way, I'll have a copy of all my media safely backed up at another site (their house), and I won't need to pay a hosting fee or anything for them to have it there. Bittorrent sync can sync huge files.
Click to expand...

I think the issue for some would be worry about general internet performance, as well as data cap issues.
 
brentsg said:
I think the issue for some would be worry about general internet performance, as well as data cap issues.
Click to expand...
1) re: data cap, do the initial backup on the LAN. Once backed up, take it over. Thus, only incremental updates will get pushed.
2) re: internet performance, get a router that supports QoS to give the BT traffic a lower priority over everything else. Also keep in mind it only sends data when data is updated. So it's only using traffic if you're adding to the collection of data.
 
I really think that you should consider a NAS and some offsite backup, whether it's CrashPlan, Amazon Glacier or some other setup.

The biggest issue I see is that what you are currently doing is a tremendous waste of storage. What do you do with the old disks? In a NAS system, like the synology, you can keep adding disks and expanding your storage until the system is full. Then, you exchange smaller disks for larger disks in the NAS to expand the system.

I understand, and completely agree, that RAID isn't a backup. However, RAID does protect somewhat against drive failures (which further utilizes older drives). Of course, that's what a true backup is for.
 
I used to have the same problem

Until I woke up one day and realize that most of the photos and videos I capture I have no interest in keeping. There may be 5% that is worth keeping.

But of the mountain of photos and videos that I have to sort through to weed out the ones that I want to keep, I don't really have the time and interest at the moment.

My new mantra is to be very selective of what I shoot, and what I keep, right from the start. If you shoot a lot, and leave it until a later date to sort things out, it's not likely that they will get sorted, unless you are stuck on an island with nothing to do for a year or so.

Quality over quantity is the key I think.
 
meistervu said:
Until I woke up one day and realize that most of the photos and videos I capture I have no interest in keeping. There may be 5% that is worth keeping.

But of the mountain of photos and videos that I have to sort through to weed out the ones that I want to keep, I don't really have the time and interest at the moment.

My new mantra is to be very selective of what I shoot, and what I keep, right from the start. If you shoot a lot, and leave it until a later date to sort things out, it's not likely that they will get sorted, unless you are stuck on an island with nothing to do for a year or so.

Quality over quantity is the key I think.
Click to expand...


Bulk of what I end up keeping (and what takes the most storage space) is HD Home movies. Unfortunately, editing video (especially HD video) takes too much time and I simply do not have the time (nor will I, not in the near future)
 
ColdCase said:
Just a note that Crashplan can't accept files larger than ~6GB. Many of my raw home movie captures exceed 6GB.
Click to expand...

I have 184 movies all between 10GB to 16GB, and they have all be uploaded and sitting in their server no problem, so I'm not sure what you're talking about. Never ran into a file size issue.
 
Outrigger said:
I have 184 movies all between 10GB to 16GB, and they have all be uploaded and sitting in their server no problem, so I'm not sure what you're talking about. Never ran into a file size issue.
Click to expand...

Agreed..

I completed my quick test today and had no issues whatsoever, uploading a file that was nearly 12GB.
 
I may have over simplified. Crashplan supports larger file sizes but throttles the data to an unacceptable rate (unless you think 1MBps is acceptable). There are servers all over the place with differing performance, those in Europe seem to be better. Below is the official email from crashplan support just last April where they indicate what I observed is typical, 1MBps seems to be their average upload speeds. They are focussed on affordability (nothing wrong with that) over performance. Just saying that crashplan is not useful for of site video backups, which is typical for all the consumer affordable providers..

## All text above this line is added to the ticket ##

Mark C. (CrashPlan Support)
Apr 11 09:43 pm (CDT)

Hello,

This is typical behavior. CrashPlan does use a lot of resources because it is constantly scanning for file changes on your system. It also needs to keep track of your files with the files on our server for data de-duplication purposes. This takes up a lot of memory.

On average our users back up to us at 1Mbps. You are seeing above average backup speed. If you happen to have a very fast internet connection, it is unlikely that backups to CrashPlan Central will be able to match your upload limit. This is not a bug - it's the nature of a shared service that is designed to be affordable. We work to ensure that everyone gets sufficient bandwidth for uploads and is able to get a reasonable speed for our very reasonable price.

Regards,
Mark

---------
Apr 11 07:49 pm (CDT)

I am trying out crashplan on a trial basis. Seems like a product I could buy but:

I have several 10GB sized files (.mov videos) that seem to stall during the upload process at about 6GB. The transfer rate is ~215 Mbps for about 5 minutes and 40% complete when the rate drops to 3.5 Mbps. The estimated completion time goes from hours to months. This happens routinely. I can pause and resume the backup, the rate is 215 plus for about 5 minutes and then drops off the cliff. I have a 30/30 service verified by speed test, but my FTP clients don't seem to have this issue. According to Activity monitor crashplan is using 117MB of RAM, Java 497MB. I have 10GB installed with about 4GB free,

Is this a typical behavior? If so I'll have to re-evaluate using the product.

This email is a service from CrashPlan Support. Delivered by Zendesk.
 
I was never throttled during my upload. i subsequently uploaded another file without throttling as well.

I have a guaranteed 5mb/s upload from my ISP. My real results from the uploads I did yesterday yielded a bit over 6mb/s performance. I also believe the bottneck was my ISP plan.

I realize that this is just one example, but perhaps your issue has been resolved. Perhaps it was a configuration issue with the client and you received poor tech support. Maybe your ISP is performing traffic shaping, which is very common.

All I know is that I have used the service for over 2 years and I haven't experienced any problems with large files. I have massive amounts of data with large files and I've recovered a lost drive from the cloud archive.

I am not saying that your issue didn't happen, obviously it did. I am saying that your experience shouldn't deter people from trying the service.
 
My Crashplan+ dataset is 1.5TB for my largest single computer. We have a total of 8 computers being backed up. Pretty much flawless. I love the service.

/Jim
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back