Time Machine vs Mirrored Raid?

[wilx]

After around 4 years, I've finally replaced my G5 with an 8-core 2.66 with 12GB RAM. This makes me very happy, albeit poor.

Having spent that much on a computer, added to the fact that I'm now starting to get more music/audio freelance work, means I'm keen to get a decent backup system going. I aim eventually to get a 1tb external drive and use time machine to backup my samples drive, but what do you reckon would be the best option for the system drive?

Right now, I have a 640gb system drive, and a 640gb blank internal hd, waiting to be raided or setup as a time machine backup.

I'm not particularly concerned about user error. I haven't really made any errors in terms of deleting important stuff that I shouldn't have, but I have experienced a couple of catastrophic hard disc failures. This makes me lean towards mirrored raid, for the peace of mind of having an instant recovery if my system drive dies.
But would software mirrored raid be reliable? And would it introduce performance issues to a system where hard disk speed is important?

Excuse my ignorance - it'd be nice to hear some opinions :)

As an aside, someone on the LogicProHelp forums has done something bonkers with Raid and partitioning - maybe I could do something similar?

"So, my 4 WD6400's look like this.
HD1:
- partition 1 (300GB) for boot
- partition 2 (300GB) for nightly backups of HD2.
- This gives me a high performance boot drive (100-120MB/sec across the entire partition)

HD2:
- partition 1 (300GB) for "work files"
- partition 2 (300GB) for nightly backups of HD1.
- This gives me a high performance work files drive where I keep all my project files (100-120MB/sec across the entire partition)


HD 3/4: This one I take advantage of apples excellent soft RAID capabilities. I partition both drives with a 200GB first and 400GB second partition.
Now - watch this. With the two 400GB partitions, i create mirrored RAID (which becomes a 400Gb logical drive). With the two 200GB partitions I create a striped RAID (which becomes a 400GB logical drive). Now I can simply back up the striped RAID to the mirrored RAID. Kind of like having a RAID 10 with only two drives.
This gives me a SUPER performance drive for all my samples, loops, etc (about 180-200MB/sec across the entire 400GB logical drive!) Samples load very fast!
"

[cmaier]

After around 4 years, I've finally replaced my G5 with an 8-core 2.66 with 12GB RAM. This makes me very happy, albeit poor.

Having spent that much on a computer, added to the fact that I'm now starting to get more music/audio freelance work, means I'm keen to get a decent backup system going. I aim eventually to get a 1tb external drive and use time machine to backup my samples drive, but what do you reckon would be the best option for the system drive?

Right now, I have a 640gb system drive, and a 640gb blank internal hd, waiting to be raided or setup as a time machine backup.

I'm not particularly concerned about user error. I haven't really made any errors in terms of deleting important stuff that I shouldn't have, but I have experienced a couple of catastrophic hard disc failures. This makes me lean towards mirrored raid, for the peace of mind of having an instant recovery if my system drive dies.
But would software mirrored raid be reliable? And would it introduce performance issues to a system where hard disk speed is important?

Excuse my ignorance - it'd be nice to hear some opinions :)

As an aside, someone on the LogicProHelp forums has done something bonkers with Raid and partitioning - maybe I could do something similar?

"So, my 4 WD6400's look like this.
HD1:
- partition 1 (300GB) for boot
- partition 2 (300GB) for nightly backups of HD2.
- This gives me a high performance boot drive (100-120MB/sec across the entire partition)

HD2:
- partition 1 (300GB) for "work files"
- partition 2 (300GB) for nightly backups of HD1.
- This gives me a high performance work files drive where I keep all my project files (100-120MB/sec across the entire partition)


HD 3/4: This one I take advantage of apples excellent soft RAID capabilities. I partition both drives with a 200GB first and 400GB second partition.
Now - watch this. With the two 400GB partitions, i create mirrored RAID (which becomes a 400Gb logical drive). With the two 200GB partitions I create a striped RAID (which becomes a 400GB logical drive). Now I can simply back up the striped RAID to the mirrored RAID. Kind of like having a RAID 10 with only two drives.
This gives me a SUPER performance drive for all my samples, loops, etc (about 180-200MB/sec across the entire 400GB logical drive!) Samples load very fast!
"

mirrored raid should not adversely affect your speed. (except insofar as you are using mirrored raid instead of striped raid, though you can even do both with raid10).

Raid does you no good if 2 disks fail, if the computer fails, if an OS or software glitch overwrites or corrupts data, etc. Also of no help if your computer bursts into flames or gets crushed by a megawombat.

I always recommend mirrored raid PLUS backup to an external (and preferably "off site" - or at least in a different part of the building) disk (which, preferably, is also mirrored raid), at least for user data.

[geoffreak]

Time machine gives you incremental backups which can help when you accidentally save over the wrong file or if all your other drives fail. RAID (1, 5, or 6) gives data redundancy which can help when a drive fails. For safety, use both.

[hayduke]

RAID 1 or RAID 5 (you'd need a RAID card for 5) is not a back-up system. but rather protects you against hard drive failure and can offer file I/O advantages. TM is a good and simple back-up strategy, but if you are as careful as you say this may not be necessary, but I still think it is wise. You might consider burning DVDs and/or storing a hard drive off site too as part of a good back-up strategy.

Software RAID1 will have a very slight performance hit, but it will protect you against a single drive failure. RAID 0 is faster, but can't tolerate a disk loss and should never be used without a proper back-up system.

[UltraNEO*]

If you need high-speed access to your data, then RAID is the way to go but it's shouldn't be used by any means as a secure backup of data, even as a RAID5, you'll only have the redundancy of one drive, there's always a chance another unit will fail; while the data been reconstructed. If anything, RAID systems should be thought of as a large scratch disk.

In my opinion it's better to have both, especially if you need large bandwidths, high performance and data security... Personally, as far as backup goes, I wouldn't trust my data to be on another mechanical device. Having physical backups on either Disc or some other media, will always be a safer bet.

[cmaier]

A lot of rampant agreement going on around here.

[UltraNEO*]

A lot of rampant agreement going on around here.

comes with experience?

[Mac Husky]

Question between: I don´t want to pay for the apple RAID card. So the only RAID I can go for by OS X seems to be RAID 0 and/or RAID 1. Right :confused:

a) using RAID 0/1 only, is it already usefull to chose optimized disks for RAID as Western Digital RE3 instead Caviar Black or (for friends of Samsung ;)) new Samsung RAID class modell H103UJ instead of Spinpoint F1 :confused:

b) what are the alternatives to the Apple RAID card to go for more RAID options :confused: (found this one (http://www.caldigit.com/RAIDCard/) in another thread) - but I think I don´t need one at all.

[UltraNEO*]

Question between: I don´t want to pay for the apple RAID card. So the only RAID I can go for by OS X seems to be RAID 0 and/or RAID 1. Right :confused:

a) using RAID 0/1 only, is it already usefull to chose optimized disks for RAID as Western Digital RE3 instead Caviar Black or (for friends of Samsung ;)) new Samsung RAID class modell H103UJ instead of Spinpoint F1 :confused:

b) what are the alternatives to the Apple RAID card to go for more RAID options :confused: (found this one (http://www.caldigit.com/RAIDCard/) in another thread) - but I think I don´t need one at all.

NO...

There are a few third party/after market cards out there, beside the Apple Raid card is over prices and performance lags alot (slowest out there!) Go search google for Mac Raid cards.. or search the forums. This topic been covered million of times....

To name a few, all about budget...

HighPoint Technologies
CalDigit
3ware
Dulce Systems
Sonnet
Promise technologies

[Mac Husky]

Go search google for Mac Raid cards.. or search the forums. This topic been covered million of times...
I guess so - yes. But as I never dealed with this topic, it is hard for me to decide, which one is best, good or favourable by reading a lot of arguments and data, that doesn´t say much to me. Sorry for that.
To name a few, all about budget...
Could you PLEASE give an advice, which one might be a good choice to go for in a Mac Pro for a reasonable price?!
I will see, if it is available in Germany. THX :)

[NoNameBrand]

"So, my 4 WD6400's look like this.
HD1:
- partition 1 (300GB) for boot
- partition 2 (300GB) for nightly backups of HD2.

HD2:
- partition 1 (300GB) for "work files"
- partition 2 (300GB) for nightly backups of HD1.

HD 3/4: This one I take advantage of apples excellent soft RAID capabilities. I partition both drives with a 200GB first and 400GB second partition.
Now - watch this. With the two 400GB partitions, i create mirrored RAID (which becomes a 400Gb logical drive). With the two 200GB partitions I create a striped RAID (which becomes a 400GB logical drive). Now I can simply back up the striped RAID to the mirrored RAID. Kind of like having a RAID 10 with only two drives.

The only trouble with that setup is that you're reading and writing from the same disks to backup the striped array to the mirrored partitions, so the disks are going to be thrashing. Better off backing up the striped array to disk 1 or 2, and that disk to disk 3 or 4. If you don't clone the striped array very often, it would probably be bearable.

So, I might do something like:
HD1: 640G: Boot 300G ; Stripe backup 300G
HD2: 640G: Data 300G ; Boot Backup 300g
HD3: 640G: Stripe 150G ; Data backup 300G ; Spare 150G
HD4: 640G: Stripe 150G ; Time Machine 450G
Striped array: 300G

Everything is backed up, and sequential clone operations won't read/write the same disk at the same time. Anything that's stored in Time Machine will be backed up again. While your sample library isn't backed up onto a mirror, I think you'd be OK. If you want a mirror, just get a 1TB drive for HD2 and mirror a 300G 'stripe backup' partition from HD1.

[cmaier]

I guess so - yes. But as I never dealed with this topic, it is hard for me to decide, which one is best, good or favourable by reading a lot of arguments and data, that doesn´t say much to me. Sorry for that.

Could you PLEASE give an advice, which one might be a good choice to go for in a Mac Pro for a reasonable price?!
I will see, if it is available in Germany. THX :)

Note that with an 09 MP, you will probably have to pu your disks in an external enclosure I you us a non-apple raid card. The new MP doesn't use cables to connect the drives. You should take the cost of an enclosure into account.

[wilx]

Thanks for all the helpful comments.

As I've only got three HDs at the mo (640gb, 640gb, 1tb) I'm thinking:

HD1: One 300gb system partition
HD1: RAID Backup for HD2

HD2: One 300gb partition for music and pictures and misc data
HD2: RAID backup for HD1 [edited to make sense :)]

...and leaving the 1tb drive as a 'normal' 1-partition sample drive for now.

Unless there's a reasonably pain-free way of getting striped raid into the setup? Would the performance increase be particularly noticable? I'm thinking maybe it could cut loading times for large orchestral Logic templates - which is something I'm doing with increasing frequency.

[NoNameBrand]

How much storage space do you need for your slow-loading templates?
RAID0 on n-disks isn't quite n*slowest drive speed, but it's still loads faster than one disk.

Why do you want HD2 backed up onto a mirrored RAID with the same disk as a component? if you lose HD2, you've got a backup of your data on HD1, and if you lose HD1, you've got the original of the data on HD2.

Do you really need the full 1TB of space for samples?

[wilx]

How much storage space do you need for your slow-loading templates?
Do you really need the full 1TB of space for samples?

At the moment, I use the 1tb drive for :

Samples
Logic projects (with all their associated audio files and edits

Currently there's about 400gb used, so I'd like to keep a reasonable amount free for future sample library acquisitions and many more logic projects.



Why do you want HD2 backed up onto a mirrored RAID with the same disk as a component? if you lose HD2, you've got a backup of your data on HD1, and if you lose HD1, you've got the original of the data on HD2.


I'm an idiot.:rolleyes:
I've edited the post so it makes more sense. The 2nd partition on HD2 would be a backup of the system drive's (HD1) data.

[NoNameBrand]

At the moment, I use the 1tb drive for [lots]

Fair enough - I was just curious. I have my TB disk divided into six partitions.

I'm an idiot.:rolleyes:
I've edited the post so it makes more sense. The 2nd partition on HD2 would be a backup of the system drive's (HD1) data.

Er, you can't RAID a single partition; do you mean the first partition of one disk and the second partition of the other are mirrored?

If so, a few things:

-You're going to have worse performance than if you have the two first partitions mirrored and the two second partitions mirrored (well, in your X configuration, everything will be as fast as the second partition array in my suggestion, while the first partition in mine will be faster)

-RAID isn't backup.

-RAID isn't backup.

- (again) RAID isn't backup. It's high availability. Absolutely use it if you can't have any downtime, but it's not backup. A proper backup solution allows your kid to accidentally delete your files and you recover them, RAID won't cover you there - your RAID will helpfully delete the mirrored files simultaneously. A better solution is to clone your drives nightly or weekly, or whatever. You will clone your errors if you aren't fast enough, but it gives you a window to recover from bad updates, horrible children, etc.

Speaking of recovery windows, Time Machine's window is long. I really think everyone should use it. Limit what it backs up to just your important files - the music you create, your email, etc. Don't have it back up your samples, apps or OS.

[wilx]

Thanks for that - I'll eventually be getting an external HD and making Time machine backups that way, but I'm far less worried about human error than mechanical failure.

So for now:

1st 640gb HD
1. System
2. RAID mirror for data


2nd 640gb HD
1. Raid mirror for system
2. Data

3rd 1tb HD - no RAID, but maybe striped and mirrored when I get another 1tb drive.

Is that what you meant? I wasn't aware that partition numbers had a bearing on the speed (as I said in the orginal post, I'm new to all things RAID) - is it something to do with having each RAID pair consist of one partition from the outside of the HD and the other partition from the centre?

[NoNameBrand]

Is that what you meant?

Yes, for speed of the RAID.

I wasn't aware that partition numbers had a bearing on the speed (as I said in the orginal post, I'm new to all things RAID) - is it something to do with having each RAID pair consist of one partition from the outside of the HD and the other partition from the centre?

Partitions closer to the edge are faster. Writing to your RAID will be the slowest of the disk write speeds of the components; reading will approach the sum of the read speeds (assuming OS X software RAID splits reads - it might not, in which case it'll read from one of the disks).

In your "X" config, you'll be writing at the speed of the second partition on both arrays, whereas in the other configuration, the first array will have a faster write speed than the second.

You might want to read the Mac Performance Guide (http://macperformanceguide.com/). It's aimed at photographers, but it's a good read.

IIRC, There are some music/audio production threads down below that reference more topical forums for your needs.

If you do want to create a striped array, make a partition of size x at the front of each of the three disks, where x is a little more than 1/3 of the space requirement. Make sure you backup this data somewhere...

[wilx]

Thanks again - the advice is much appreciated.

Finally... (I think!)

With striped RAID, I'm assuming it would make most sense to have my sample drive as striped, since that's where all the files need to be streamed from when I'm in Logic.

So I could split the two 640gb drives into
1.
200gb RAID stripe
200gb system
200gb RAID mirror (data)
2.
200gb RAID stripe
200gb RAID mirror (system)
200gb data


....and have all my sample data on the 600gb stripe disc?

That'd leave me with an 800gb partition on the 1TB disc - dunno how that could fit into the setup!


This is making my brain hurt, but I'm sure it'll pay off in the long run - thanks again for helping out. :)

[wilx]

Ah, sod it. This seems simpler:


1st 640gb HD
1. System
2. RAID mirror for data


2nd 640gb HD
1. Raid mirror for system
2. Data

------------------

When I get a 4th drive, I'll set that up with my third drive (both 1tb) as a striped disc.

And use Time machine for backups to an external HD or two.


This is assuming the 1tb disc (audio/logic projects/video) would benefit most from striped RAID.


EDIT: after all this, I've read on some music forums that partitioning and RAID is far less of a good idea for audio stuff than video, so I might end up just having the 2nd 640gb disk as a time machine internal backup! At least I'm now better informed about RAID for next time...

[m1stake]

Remember, RAID1 will save you from disk error. RAID1 will not protect you from user stupidity (i.e. you accidentally lose something). Do not forgo traditional backups for RAID1. Do both!

[hazelbrook]

I'm new to mac OS, just own a G5. I notice in the Disk Utilities, there is an tab called RAID. What is this? which level of RAID does OSX supports? If I have 2 internal HD, can I set HD1 mirrors HD2?

Thanks guys!!!

[Tesselator]

Ah, sod it. This seems simpler:


1st 640gb HD
1. System
2. RAID mirror for data


2nd 640gb HD
1. Raid mirror for system
2. Data

------------------

When I get a 4th drive, I'll set that up with my third drive (both 1tb) as a striped disc.

And use Time machine for backups to an external HD or two.


This is assuming the 1tb disc (audio/logic projects/video) would benefit most from striped RAID.


EDIT: after all this, I've read on some music forums that partitioning and RAID is far less of a good idea for audio stuff than video, so I might end up just having the 2nd 640gb disk as a time machine internal backup! At least I'm now better informed about RAID for next time...

That's right. Audio guys very often say No RAID! But I dunno why. In my experiments audio recording onto a RAID 0 preforms just fine. <shrug> I guess they know more them me tho. I'll assume they do.

But what are you talking about here? You're partitioning separate single drives and then using those partitions in a RAID set??? Is that even possible? If it is somehow possible that sounds pretty flakey. I wouldn't do it! You RAID 0 stripe first and THEN partition. And with your limited number of drives RAID 1 is a terrible waste of resources for just about zero benefit.

If it was me with your available 3 drives I would RAID 0 the two 640GB drives and then run a media tester that tells you your average R/W speeds across the entire platter. (1.28TB) From there you can make an intelligent decision about where to set your partition points. Usually the 1st 20% is 2x faster than the last 20% with the 60% in the middle being well, in the middle.

So usually three partitions. 1 for the fastest R/W throughput (DTD Audio recording and etc.). 2 for project editing, and 3 for clip FX and instrument storage. Or something like that. You know which you use most and what would benefit from the speedy partitions better than I.

Then finally the 1TB drive gets partitioned into two parts 200GB ~ 300GB in partition 1 for the System Boot and partition 2 for your daily project backups and etc.

And honestly if it were me I would buy two more drives. One more 640GB drive so you can do the above 1, 2, 3 partitions on a much faster RAID set and then a 1.5TB external drive that I would use to back EVERYTHING up weekly or as you see fit to schedule - maybe Time Machine is good for this? There seems to be some debate.

The 1.5TB external should b kept in a powered down state most of the time when not backing up.

[pfleischman]

Works great, except when you run disk utilities like Onyx. For some reason the mirror *can* degrade when you do maintenance on it. This has happened to me once since I've set it up about 10 months ago. I have a third drive (same 1TB size) as a Time Machine Drive, just in case. Also I burn my data to DVD. But overall I like the read speeds and lack of worry if one drive dies. Who want's to re-install EVERYTHING! :rolleyes:

[NoNameBrand]

But what are you talking about here? You're partitioning separate single drives and then using those partitions in a RAID set??? Is that even possible? If it is somehow possible that sounds pretty flakey.

Of course it's possible. Software RAID is fantastic. I have the first 15GB of each of the four disks on my Mac Pro in a striped array for Photoshop scratch. I had a similar setup on my G5 for years. Works very, very well.

~295MB/s read/write speed - two of the component disks can only hit ~75MB/s on their own. Eventually they'll be replaced by newer, faster disks and I'll see closer to 400MB/s.

[brendon2020]

my setup is 3 drives, one for time machine and then the os mirror'd across two drives. I use the software raid that came with leopard and haven't had any issues or noticed any slowdowns.
The raid software is very limited but once its setup never had any issues with it.

[UltraNEO*]

I guess so - yes. But as I never dealed with this topic, it is hard for me to decide, which one is best, good or favourable by reading a lot of arguments and data, that doesn´t say much to me. Sorry for that.

Could you PLEASE give an advice, which one might be a good choice to go for in a Mac Pro for a reasonable price?!
I will see, if it is available in Germany. THX :)

If needed, there's a company here in the UK that will rent/lease (http://rentaraid.co.uk) you a RAID system, it might be a good way to try the various cards. But that's might not be the most economical solution. Really depends on your work load, requirements and budget... it can become fairly costly if you go down the hardware RAID route but your data will be safer!!

Initially I looked into the RocketRAID but finding a supplier with stock proved to be impossible at the time. Eventually I settled for the CalDigit RAID card cause it's relatively fast, cost effective, expandable (have to use their own external boxes) and most importantly, does what it says in the box!!! :)

Now I think various people on here would say the card originally had teething issues and they would be right. However, for the duration that I've been using the hardware, I've not encountered any hardware glitches yet, except one failed drive:(


My MP have the following RAID5 configs, primary used for editing and processing HD Video content, AfterFX and some graphics for print. Typical streams varies from 500MB to 8Gb or more!

　Optical slot 1 - Panasonic 8x Blu-ray Burner
　Optical slot 2 - System - WD VelociRaptor 300Gb - WD3000HLFS

RAID5 Scratch disk (totalling 2.79TB, redundancy of 1)
　HD 1 (slice 1) - WD RE3 1TB - WD1002FBYS
　HD 2 (slice 2) - WD RE3 1TB - WD1002FBYS
　HD 3 (slice 3) - WD RE3 1TB - WD1002FBYS
　HD 4 (slice 4) - WD RE3 1TB - WD1002FBYS

The MP still has the onboard iPASS SATA interface available for further expansion but those drive wil have to be housed and powered externally.

If you wanna go with hardware and need some professional advice. Then there's a user on those forums called "nanofrog".
I'd recommend chatting to him about choosing cards but we basically agree on the above setup, when it comes to drives (except the BD-RE):rolleyes:

[sushi]

I clone my internal HD (500GB) to alternate external HDs every week via FW800.

I also am using Carbonite and ADrive for offsite or cloud storage/backup.

The problem with only local backup is that you could loose your computer and backup at the same time due to an earthquake, fire, flood, theft, etc. Suggest considering offsite/cloud storage additionally to your local mirror/RAID/external hard drive back up.

[UltraNEO*]

I clone my internal HD (500GB) to alternate external HDs every week via FW800.

I also am using Carbonite and ADrive for offsite or cloud storage/backup.

The problem with only local backup is that you could loose your computer and backup at the same time due to an earthquake, fire, flood, theft, etc. Suggest considering offsite/cloud storage additionally to your local mirror/RAID/external hard drive back up.

Yes.. Always backup valuable data, never chance it.
Though at worst, there's always data recovery centres, unfortunately it could be costly for the domestic user.

[Tesselator]

Of course it's possible. Software RAID is fantastic. I have the first 15GB of each of the four disks on my Mac Pro in a striped array for Photoshop scratch. I had a similar setup on my G5 for years. Works very, very well.

~295MB/s read/write speed - two of the component disks can only hit ~75MB/s on their own. Eventually they'll be replaced by newer, faster disks and I'll see closer to 400MB/s.

Hmmm, shows ya what I know. This is the 1st time I'm hearing of it at least that I can recall. And why are you calling it "software RAID"? Is this not possible with a dedicated RAID controller or something?

[Mac Husky]

(...)
Thanks a lot for your detailed explanations.

If it is right, what cmaier wrote
Note that with an 09 MP, you will probably have to pu your disks in an external enclosure I you us a non-apple raid card. The new MP doesn't use cables to connect the drives. You should take the cost of an enclosure into account.
than RAID5 is not making much sence in the workflow by using third party?!

I could make a backup easier with Time Machine on a big HD inside the Mac.
And a second one (e.g. weekly) to an external drive via FW800.

Further Question:
Using a WD VelociRaptor as system drive, would it be usefull to build a partition including it in a RAID0 scratch for Photoshop?
Or should one leave the system disk beeing system disk at all and better build RAIDO scratch using disk 2 and 3 alone?

[sushi]

Though at worst, there's always data recovery centres, unfortunately it could be costly for the domestic user.
As long as you don't have a mechanical failure that damages the disk.

In these cases, even the data recovery folks will be limited as to what they can recover.

[NoNameBrand]

Hmmm, shows ya what I know. This is the 1st time I'm hearing of it at least that I can recall. And why are you calling it "software RAID"? Is this not possible with a dedicated RAID controller or something?

In my experience with HP & Dell servers, you have to use whole disks in your sets with a RAID controller, and then partition on top of that, which isn't as useful. Intel has something called "Matric RAID" which is built into some of their chipsets that allows different RAID levels on the same disks, but it's not your typical hardware RAID controller with its own processor or battery backup.

[UltraNEO*]

As long as you don't have a mechanical failure that damages the disk.

In these cases, even the data recovery folks will be limited as to what they can recover.

Very True...

but with a 4disc RAID5 failure, there's still a chance to recover lost data - there's the redundancy of one, remember?