Suggestions for setting up a RAID system?

[stagi]

I need to setup a RAID 1 system for my image files for this coming up year. I will need about 1TB of space for the year and am pretty new to setting up a RAID config. (in the past have just been using 1 large external drive and backing it up to another manually).

Any suggestions for what brand/type of drives to buy and when setting up a RAID setup do all of the drives have to be the exact same?
Anyone have expierence with the buffalo stations, seems easy since they are already setup:
http://www.buffalo-technology.com/products/product-detail.php?productid=146&categoryid=19
or should I buy an enclosure and drives seperate?

Any help is appreciated!

 

[alFR]

I'd consider a RAID 5 array as well (basically you need a group of at least 3 disks, in the event of failure "the parity blocks from the surviving disks are combined mathematically with the data blocks from the surviving disks to reconstruct the data on the failed drive "on the fly" - from wikipedia). The advantage of this over RAID 0 is that you get more storage available for the same number of drives.

The Infrant ReadyNAS will do RAID5, as will the Buffalo Terastation. Both of those will do other stuff like serving printers too and IIRC both have gigabit ethernet.

 

[Sherman Homan]

Buffalo's Terastations are great little boxes, but I don't think you want RAID Level 1. That is mirrored redundancy which will not give you any speed increase. As alFR said, I think you want Level 5.
My question for you is do you really want to set up a network server? If you working from a single computer you will get faster access speeds with firewire.

 

[weldon]

Since you don't sound interested in building your own fileserver, I would recommend waiting to see what's in store with Leopard before making a commitment to a big NAS box (Infrant or Buffalo). Time Machine and ZFS will introduce some new approaches to managing redundant copies of files that may change the solution that would work best for you.

Also, keep in mind that mirroring or RAID5 is not a replacement for backup. RAID5 can save from a single drive failure, but it does nothing to help you if the file becomes corrupted (although ZFS might). If you need a TB of storage, you might look into a My Book Pro firewire drive as an interim solution for now and a backup location when Leopard comes out. The 500GB drive is about $230 and the 1TB drive is about $450. It comes with decent backup software, although SuperDuper is worth a try too. That's what I prefer.

 

[CanadaRAM]

You are asking about two different things:
The Terastation is a Network Attached Storage unit, or NAS

RAID is an array of drives that are combined to provide either speed, or redundancy, or very high single volume capacities.

The two are not related, although a NAS unit can incorporate RAID in its setup.

You need to clarify what your goals are.
If you are looking for outboard network storage, that is accessible by more than one machine on the network, you want a NAS.

If you have a NAS, forget about speed gains from RAID striping, because your limiting factor is the Ethernet and the network overheads.

If you want redundancy, you need to look at a RAID level 1 or higher in your storage device,

If you want large volume support, ask yourself - do you really need more than 750 Gb in a single volume? Can you divide your library across multiple volumes?

RAID 0 or disk spanning provides larger volume size, at the expense of no redundancy and much greater risk. This is what you usually get with 2-drive "Big" drives.

RAID 1 gives you redundancy, at the expense of cutting the usable size in half. Two 500 Gb drives Raid 1 will net 500 Gb of storage.

RAID 5 (or some other variation) allows you to use 3, 4 or more drives, with one drive dedicated to redundancy. So you lose 33% or 25% of your total disk space, but can recover from the failure of a single drive. RAID 5 systems are expensive because of the controller needed, the enclosure, and the multiple disks.

What are your goals?

Listen to weldon, ^ RAID != backup.

 

[stagi]

Thanks for the info, for a raid 5 system do you still have a copy of your data? I read the wiki site explanation but didn't fully understand the differences.
Also I may have a few more workstations that would need to access the files later on in the year as things expand so having the drives networked would be a good benefit, however how big is the speed loss compared to connecting it via firewire?

and last question what is ZFS (in basic terms please) and how does that help with backup? Thanks!

 

[stagi]

You are asking about two different things:
The Terastation is a Network Attached Storage unit, or NAS

RAID is an array of drives that are combined to provide either speed, or redundancy, or very high single volume capacities.

The two are not related, although a NAS unit can incorporate RAID in its setup.

You need to clarify what your goals are.
If you are looking for outboard network storage, that is accessible by more than one machine on the network, you want a NAS.

If you have a NAS, forget about speed gains from RAID striping, because your limiting factor is the Ethernet and the network overheads.

If you want redundancy, you need to look at a RAID level 1 or higher in your storage device,

If you want large volume support, ask yourself - do you really need more than 750 Gb in a single volume? Can you divide your library across multiple volumes?

RAID 0 or disk spanning provides larger volume size, at the expense of no redundancy and much greater risk. This is what you usually get with 2-drive "Big" drives.

RAID 1 gives you redundancy, at the expense of cutting the usable size in half. Two 500 Gb drives Raid 1 will net 500 Gb of storage.

RAID 5 (or some other variation) allows you to use 3, 4 or more drives, with one drive dedicated to redundancy. So you lose 33% or 25% of your total disk space, but can recover from the failure of a single drive. RAID 5 systems are expensive because of the controller needed, the enclosure, and the multiple disks.

What are your goals?

Listen to weldon, ^ RAID != backup.

My goals are to have a storage system for image files (about 800 GB estimated for the year), that can be accessed from a few computers in the office and that is backed up in case there is a drive failure. From your info I am thinking a NAS Raid 1 would be good?

 

[Sherman Homan]

My goals are to have a storage system for image files (about 800 GB estimated for the year), that can be accessed from a few computers in the office and that is backed up in case there is a drive failure. From your info I am thinking a NAS Raid 1 would be good?

Ah, you do want an NAS like the Terastation or the Infrant. But you do not want Level 1, you want Level 5. And as others have said, any time you use the word RAID you must also say BACKUP!

 

[jeremy.king]

But you do not want Level 1, you want Level 5.

Why do you say this? If all he wants is redundancy, RAID 1 should be fine.

 

[alFR]

RAID 5 doesn't provide a copy of your data, it provides the ability to reconstruct it if one of the drives in the array fails. Basically you pull out the broken drive, stick in a new one and it chugs away for a while rebuilding the data that was "lost".

All RAID protects you from is disk failure, not user error. Neither a RAID 5 nor a RAID 1 array will protect you if you accidentally delete a file, make an editing mistake etc. - to recover from either of those you would need a backup on a different volume, which could be on (ideally?) a completely separate disk or another volume on the same array (or on a DVD etc.). The Infrant drive allows you to plug in a USB hard drive and will backup the contents of the array to that: don't know about the Buffalo.

 

[weldon]

Thanks for the info, for a raid 5 system do you still have a copy of your data? I read the wiki site explanation but didn't fully understand the differences.
Also I may have a few more workstations that would need to access the files later on in the year as things expand so having the drives networked would be a good benefit, however how big is the speed loss compared to connecting it via firewire?

and last question what is ZFS (in basic terms please) and how does that help with backup? Thanks!

RAID5 does not keep copies of your data. It works by keeping a checksum of your data distributed across all the drives in the array. If you were to lose one drive in the array, it could rebuild the info from the information stored on the remaining drives. RAID5 does this with fixed length stripes across the entire array, which means it has to calculate the checksum for every stripe in the array. If you lose two drives, your data is toast. You replace the failed drive in the array and the RAID system will rebuild the data from the info on the other drives. It's very slow while rebuilding, but good as new when done. RAID6 does double parity at the cost of the equivalent of two drives lost to parity info. The benefit is that you can survive two drive failures before losing data.

If an application does something funky and corrupts your file, that file is toast because the only copy is corrupt. That copy is safe from a single drive failure, but it's still corrupt and unusable. If you delete a file, it's gone. If you make an editing mistake, the original is gone.

ZFS is a filesystem (FAT32, HFS+, NTFS, UFS, etc.) and a volume manager (RAID, etc.) combined in one. ZFS has a feature called RAIDZ to create storage pools or arrays of disks. RAIDZ can do single or double parity like RAID. It manages the parity info more efficiently so that it only has to rebuild the data that is missing rather than the entire volume. ZFS also has built-in checksums and "copy on write" for all files (RAIDZ or not) so that it can detect when a file is written incorrectly to disk and automatically correct the error. This can save you from some problems with corrupt files. It also closes a RAID5 write hole where you can lose data on a failure during a write operation.

ZFS also supports snapshots which is a way of keeping copies of your data frozen at a certain point in time. It does this very efficiently by keeping the data in place and then writing changes while keeping the "frozen" copy in place. You can "roll back" changes this way to the point when you took the snapshot. Useful for recovering a corrupted file, misedited file, etc.

My goals are to have a storage system for image files (about 800 GB estimated for the year), that can be accessed from a few computers in the office and that is backed up in case there is a drive failure. From your info I am thinking a NAS Raid 1 would be good?

Because you want to access these files from mutiple computers, you should be looking at a NAS. The Infrant box is very nice and can do RAID5 or X-RAID for some redundancy. The RAID on this box will save you if you lose a single drive in the array. You could also backup the Infrant to a firewire drive attached to a computer.

It's also possible that the Airport Extreme with a USB drive attached could be a good solution. I would wait for more user reports before committing to that route though.

 

[CanadaRAM]

It's also possible that the Airport Extreme with a USB drive attached could be a good solution. I would wait for more user reports before committing to that route though.

Oh no -- Airport Extreme + USB is WAAAY too slow for accessing image files.

One thing - if you are using a NAS for network file storage, you will be attaching to it from Macs and PCs as a SMB server. This imposes some limits on Mac users in terms of file names, and perhaps individual file sizes (IDRRN)

If yours is an all-Mac office, you have the option of turining a machine like a G4 tower into a fileserver. This would allow you to address the files as an AFP (native Mac filing system) volume. It would also allow you to run Mac software in the background on it, like Retrospect, for doing backups.

 

[stagi]

Oh no -- Airport Extreme + USB is WAAAY too slow for accessing image files.

One thing - if you are using a NAS for network file storage, you will be attaching to it from Macs and PCs as a SMB server. This imposes some limits on Mac users in terms of file names, and perhaps individual file sizes (IDRRN)

If yours is an all-Mac office, you have the option of turining a machine like a G4 tower into a fileserver. This would allow you to address the files as an AFP (native Mac filing system) volume. It would also allow you to run Mac software in the background on it, like Retrospect, for doing backups.

Everything here is a mac, and I do have a old 400Mhz G4 that I don't use anymore. Is there a site that gives more info on how to setup a computer to be a file server, that is something totally new to me. Thanks,
Mark