Recommend a Storage Solution for the Future!

[fivepoint]

After filling my brand new 750GB iMac up with home videos and my ripped DVD collection, I am looking for a viable storage solution, as I am sure many of you are too. The way I see it, the more popular DVD ripping and home movie editing become, the more people are going to be having th is problem.

My question to you is this: What is the best solution for the average mac user who needs considerable external storage, which is expandable for the future and has the capability to back itself up?

I've done a BIT of research on RAID, and have familiarized myself with the basics. I have also seen a few videos on the drobo, which seems to get great reviews everywhere. But, I want your opinion. For everyone with an ever-expanding iTunes library and iMovie folder that will be searching this very topic... what is your advice? What rules-of-thumb should we follow? Which 'traps' should we look out for? Can the average consumer handle running a RAID array?

Thanks for your input!

 

[nplima]

Hi

If I were in your shoes, I'd probably think of having a separate computer for storage only. Something like a relatively silent tower with a good hard disk controller and room for expansion for an additional hard disk controller, ie: something with PCI slots, not another iMac.

I would keep this machine running with a variety of Linux with only basic components on. This machine would not be used for anything else but serving files. I'd keep the operating system on one small hard disk and /home on a BIG disk with x capacity costing y. Every other year, as new hard disks being sold carry 2 x or 4 x for the same y price, I would pick up what's inside that /home, transfer to a new disk, replace it and wait another 2 years to start over again. External hard drives could also be attached to this machine.

I've seen a post here on these forums showing how someone set up a powermac G4 to this job, which had the advantage of having OS X instead of having to learn how to set up your server on Linux. The disadvantage was that G4s were already a dead architecture and in the future getting new hard disk controller boards would prove more complicated and a limitation, compared to using just about any hardware on linux.

 

[fivepoint]

nplima,
Thank you so much for taking the time to respond. Your suggestion is duly noted, however... I think I need a simpler solution. While I feel confident that I COULD do this, I just don't think I want to... and I would feel more comfortable with a single-computer solution.

There HAS to be a simple mass-storage device out there which backs itself up, right? Do you have any other suggestions?

Thanks!

 

[ButtUglyJeff]

If Drobo+Droboshare was faster, I would say its nearly perfect. Its very user friendly, very easy to upgrade, and its fully Time machine compatable. If you don't want to have to manage something, and speed isn't a priority, the Drobo is a great choice. I don't regret my purchase at all.

 

[fivepoint]

Thanks for the input, Jeff. I think the Drobo is top on my list as of now.


Someone in a different forum mentioned something interesting on this topic... how about using Apple's software RAID in Disk Utility? Has anyone here used that? How has it worked for you? Can you give me the basic run-down of how it works, what the benefits and negatives are?

Thanks!

 

[Consultant]

Drobos are very expensive for what it does.

For years, I use a PowerMac G4 with gigabit ethernet with currently 5 HDs in it (you can fit 6 if you remove the dvd drive). It's a lot more flexible than a NAS.

There are many PCI cards that works with IDE and SATA drives. I do expect it to run for years to come.

Runs perfect 24/7 with one processor at 100% doing Folding, and the other processor 50%.

 

[fivepoint]

Drobos are very expensive for what it does.

For years, I use a PowerMac G4 with gigabit ethernet with currently 5 HDs in it (you can fit 6 if you remove the dvd drive). It's a lot more flexible than a NAS.

There are many PCI cards that works with IDE and SATA drives. I do expect it to run for years to come.

Runs perfect 24/7 with one processor at 100% doing Folding, and the other processor 50%.

That solution is limited to Mac Pros only though, right? What about iMacs, notebooks, etc?

 

[Consultant]

That solution is limited to Mac Pros only though, right? What about iMacs, notebooks, etc?

PowerMac G4 can be bought from $100 and up, but like anything pre-owned, you need to know what to look for. THE SOLUTION IS TO USE IT AS A SERVER.

With the proper set up, you can connect to your Mac from anywhere.

Any Mac desktop or laptop, or even PCs can connect to any Mac if you set it up for that purpose.

 

[ButtUglyJeff]

FYI - I spent $900 for the Drobo, plus Droboshare, plus 2 500GB Seagate drives. It took literally 10 minutes to be up and running and add two more hard drives, then another 2 hours to transfer all data onto it.

I agree that a G4 would make a cool home server, I just don't think I would enjoy the process of getting it there.

 

[nplima]

hi

I just had a look at Drobo and I would disagree with the idea it is expensive. What I suggested, to use a standard PC starts at £200 while Drobo goes for £329... that's not enough of a difference considering the set up time and quality assurance.

If people want simple and are not too worried about performance, I'd go with a batch of Western Digital MyBook (I only have one and it's 33% full); for ever-expanding space, a storage appliance like Drobo or a standard PC set up as storage server.

Using a PowerMac I think would be a wate of money. A new one is too expensive, a used one is too risky in the long term because of hardware limitations. We just don't know what disk controllers wil be used 5, 10 years from now but we can be quite sure that finding hardware add-ons for a G4/G5 will be tricky.

 

[fivepoint]

Good comments guys, thanks.

How about the software RAID solution I mentioned a few posts up? Can I just get a couple of external HDs and connect them through Disk Utility to mirror eachother?

Has anyone here done that? Is it more difficult than it sounds?

 

[prostuff1]

Good comments guys, thanks.

How about the software RAID solution I mentioned a few posts up? Can I just get a couple of external HDs and connect them through Disk Utility to mirror eachother?

Has anyone here done that? Is it more difficult than it sounds?

Don't take this the wrong way but that sounds like a fubar situation waiting to happen, espicially since you are trying to do it to external drives.

The Drobo's look nice and work great but when you can build something that will do virtually the same thing for much less, well i would go with the much less solution.

Anyway, your best bet is to seriously consider building some sort of home media server. This can be done pretty cheaply depending on how deep you want to get in and how "easy" you want to make certain things. I outlined a lot of this in this thread so i suggest looking there.


I am in the process of building my media server right now. It is all together I just need to get one more HD to put in it and then i can start transferring stuff.

 

[fivepoint]

Don't take this the wrong way but that sounds like a fubar situation waiting to happen, espicially since you are trying to do it to external drives.

The Drobo's look nice and work great but when you can build something that will do virtually the same thing for much less, well i would go with the much less solution.

Anyway, your best bet is to seriously consider building some sort of home media server. This can be done pretty cheaply depending on how deep you want to get in and how "easy" you want to make certain things. I outlined a lot of this in this thread so i suggest looking there.


I am in the process of building my media server right now. It is all together I just need to get one more HD to put in it and then i can start transferring stuff.

Prostuff,
Thank you for taking the time to respond. I am, however, a little confused about your statement. I stated earlier that building one from scratch was not something I would like to do. I don't even posess the skills necessary.

What I am looking for is something with no 'processing power', just something to store data that won't fit on my computer. I need a external HD solution such as the Drobo, or possibly two HDs connected via Disk Utility mirroring eachother (I don't know enough about this to know if it is even possible). I need something that takes minimal 'management' and is basically 'plug and play'.

BTW, what is so FUBAR about having two External HDs hooked up to your computer? You don't think it will work for basic storage, or do you just think it won't look good? Because if that's the case, I have a desk which will hide them anyway. (check sig)

I am honestly interested in your input... you seem very knowledgeable on the topic (I TRIED to read your info from the other thread) and I would love to hear what you think... just keep the suggestions within my skill level!

 

[prostuff1]

Prostuff,
Thank you for taking the time to respond. I am, however, a little confused about your statement. I stated earlier that building one from scratch was not something I would like to do. I don't even posess the skills necessary.

What I am looking for is something with no 'processing power', just something to store data that won't fit on my computer. I need a external HD solution such as the Drobo, or possibly two HDs connected via Disk Utility mirroring eachother (I don't know enough about this to know if it is even possible). I need something that takes minimal 'management' and is basically 'plug and play'.

BTW, what is so FUBAR about having two External HDs hooked up to your computer? You don't think it will work for basic storage, or do you just think it won't look good? Because if that's the case, I have a desk which will hide them anyway. (check sig)

I am honestly interested in your input... you seem very knowledgeable on the topic (I TRIED to read your info from the other thread) and I would love to hear what you think... just keep the suggestions within my skill level!

I guess my general thought was that if you are filling up a 750GB drive already then you probably will end up having a lot of externals. The server idea centralizes all of that kinda stuff.

I guess it depends on how much external storage you want and how much you think you might need. The central computer idea allows you to expand as your needs grow. If you don't mind spending the money then something like a Drobo will probably be your easiest and simplest solution. You can also get something like this which is a pre-built NAS that all you do is put the drives in. It is similar to the Drobo just not quite as pretty.

And on the Disk Utility thing about RAIDing external drives i would avoid that. I have never done it so i can not say how well it will work, but I would just set something up like SuperDuper/Carbon Copy Cloner to duplicate the drive over to the other.

 

[nplima]

Can I just get a couple of external HDs and connect them through Disk Utility to mirror eachother?

Has anyone here done that? Is it more difficult than it sounds?

HI,

Not having done anything of that sort, I can imagine it would be unbearably slow. PCI bandwidth is quite higher than what you get from the USB port to the rest of the computer.

There are devices that have their own hard disk controller with RAID in the box and the USB connection is from that box to the PC. Pile up a bunch and it will work.

I've been using WD mybooks and I understand that the firewire version can be daisy-chained. this way you only use one connection to your PC. Try this:
http://www.wdc.com/en/products/products.asp?driveid=410

 

[fivepoint]

I guess my general thought was that if you are filling up a 750GB drive already then you probably will end up having a lot of externals. The server idea centralizes all of that kinda stuff. I guess it depends on how much external storage you want and how much you think you might need. The central computer idea allows you to expand as your needs grow. If you don't mind spending the money then something like a Drobo will probably be your easiest and simplest solution. You can also get something like this which is a pre-built NAS that all you do is put the drives in. It is similar to the Drobo just not quite as pretty.

I think that 1-2TB would be good enough for me for the next year or two. Possibly 2-3TB in the next 4 years. I could buy a simple WD 2TB MyBook World II, but they cost almost as much as the drobo, and the functionality and adjustability is considerably less. The server idea does centralize everything... but I shouldn't ever need more than 1-2 externals at any one time, I would think.

Would the server computer really be that much cheaper? I mean, you have to buy the box, buy all of the individual hard drives, make sure it has everything for networking, etc... I'm sure it is, but since I don't even have the capability to build one, the point is moot...

I suppose the Drobo would give me expandability closer to the 'server' but is a bit more expensive too.

And on the Disk Utility thing about RAIDing external drives i would avoid that. I have never done it so i can not say how well it will work, but I would just set something up like SuperDuper/Carbon Copy Cloner to duplicate the drive over to the other.

Good recommendation on the SuperDuper/Carbon Copy Cloner. I will definitely consider that route. At the very least, I could use that as a short-term solution!

 

[fivepoint]

HI,

Not having done anything of that sort, I can imagine it would be unbearably slow. PCI bandwidth is quite higher than what you get from the USB port to the rest of the computer.

There are devices that have their own hard disk controller with RAID in the box and the USB connection is from that box to the PC. Pile up a bunch and it will work.

I've been using WD mybooks and I understand that the firewire version can be daisy-chained. this way you only use one connection to your PC. Try this:
http://www.wdc.com/en/products/products.asp?driveid=410

Nplima,
Ok, I had just looked at those WD hard drives... but the world edition instead of the Studio edition. I looked at the world because they have RAID 1 instead of RAID 0 which the Studio version has. 0 does me no good because it does not back itself up. Are you saying that if you daisy-chained several RAID 0 WD drives together, they would act as independent RAID 1 drives? 1 + 0, is that what you're saying?

 

[prostuff1]

Would the server computer really be that much cheaper? I mean, you have to buy the box, buy all of the individual hard drives, make sure it has everything for networking, etc... I'm sure it is, but since I don't even have the capability to build one, the point is moot...

If 1-2 TB is your limit for now then I guess you could just do externals. I am just partial to the server idea as it allows me to move everything (like music, movies, DVD storage, etc) off the local computer and allow any others that might be on the network to get to it.

In that other thread i outlined the final prices for the parts and i think it cam to like ~$350. And this is for a fully functional server. If you did some searching around for prices and bought when you saw deals you could get it cheaper. I know the one i just put together was ~$300 for everything NOT including the drives. You would have to put it all together and get it up and running; I guess it is just how much value you put on your time. I enjoy doing this kinda stuff so I don't factor the hours i spend getting it to work into the "budget." The majority of the time it is not hard to get this stuff up and running, especially if you choose to go with a Windows distribution. You can also go the Linux route which may or may not be a little bit harder. I guess the main thing is to just do some reading and see what will work best.

 

[ab2650]

The price may be above what others have mentioned but I think your request for "future proof" warrants a look at the Netgear (used to be Infrant) ReadyNAS.

I'm shocked to see the price has gone way up since I've bought it, but it certainly fit my needs:
1) Always on NAS (Network Attached Storage, accessible from multiple machines)
2) Easily upgradeable/expandable by adding extra disks.
3) Provides redundant storage (RAID 0, 1, 5, or their proprietary RAID-5.)
4) Has built in tools to run rsync backups for an additional level of data security.

For example my ReadyNAS is setup with four 500GB drives in RAID-5 which gives a ~1.5TB volume and can sustain failure of one disk with no data loss (two would be devastating however). For that reason, I have two 500 GB external USB drives that I run rsync on every night, and take one off-site alternating months. (I've setup rsync to only backup the important files that are irreplaceable to the external drives.)

If you're looking to drop a few bucks but really be "future proof" a ReadyNAS with two 1-TB drives (leaving 2 spaces available for when prices drop later) would work like a charm.

Granted you could build nearly the same system with a standard beige-box and linux, what your money gets is a TINY form factor and the ease of not having to configure and administer something so heavily.

It should be noted however that my primary concern was data integrity and NAS, not storage space, for which I was willing to pay a premium. RAID is a no-brainer once you've set it up, but you should be very cautious about doing it in software (you start to lose some benefits). And unless you're RAIDing for redundancy, you drastically increase your chances of total data loss.

 

[ButtUglyJeff]

This is the time I'd like to remind everybody that you don't need like drives in a Drobo. They can all be different. I currently have 2 - 500GB and 2 - 160GB hard drives for 760GB of redundant storage. When 1TB drives go down. I'll get one and simply pull out a 160, while the Drobo is on and doing its thing. And slide in a 1TB. Drobo will then manipulate the data, and then I'll have 1.1TB of protected storage, then 1.8TB when I replace the second 160.

I'm not anti RAID box, I just don't think its as "apples to apples" that people are assuming.

 

[nplima]

Nplima,
[...]Are you saying that if you daisy-chained several RAID 0 WD drives together, they would act as independent RAID 1 drives? 1 + 0, is that what you're saying?

no, no, no... I did not even take RAID into consideration. What I was saying is that with these devices you have a single connection to the PC (firewire) and the 1st drive connects to the 2nd drive, the 2nd drive to the 3rd, etc.
This is quite useful for iMac or laptop users who do not want to have a 2nd computer to serve files and also have a limited number of USB ports available.
-------

and now, for the most successful post ever made on these forums:

get a Windows Home Media Server
http://www.microsoft.com/windows/products/winfamily/windowshomeserver/buy.mspx

 

[fivepoint]

If 1-2 TB is your limit for now then I guess you could just do externals. I am just partial to the server idea as it allows me to move everything (like music, movies, DVD storage, etc) off the local computer and allow any others that might be on the network to get to it.

In that other thread i outlined the final prices for the parts and i think it cam to like ~$350. And this is for a fully functional server. If you did some searching around for prices and bought when you saw deals you could get it cheaper. I know the one i just put together was ~$300 for everything NOT including the drives. You would have to put it all together and get it up and running; I guess it is just how much value you put on your time. I enjoy doing this kinda stuff so I don't factor the hours i spend getting it to work into the "budget." The majority of the time it is not hard to get this stuff up and running, especially if you choose to go with a Windows distribution. You can also go the Linux route which may or may not be a little bit harder. I guess the main thing is to just do some reading and see what will work best.

Thanks for the info. I am about 100% sure I need a more simple solution with minimal management involved.

The price may be above what others have mentioned but I think your request for "future proof" warrants a look at the Netgear (used to be Infrant) ReadyNAS.

I'm shocked to see the price has gone way up since I've bought it, but it certainly fit my needs:
1) Always on NAS (Network Attached Storage, accessible from multiple machines)
2) Easily upgradeable/expandable by adding extra disks.
3) Provides redundant storage (RAID 0, 1, 5, or their proprietary RAID-5.)
4) Has built in tools to run rsync backups for an additional level of data security.

For example my ReadyNAS is setup with four 500GB drives in RAID-5 which gives a ~1.5TB volume and can sustain failure of one disk with no data loss (two would be devastating however). For that reason, I have two 500 GB external USB drives that I run rsync on every night, and take one off-site alternating months. (I've setup rsync to only backup the important files that are irreplaceable to the external drives.)

If you're looking to drop a few bucks but really be "future proof" a ReadyNAS with two 1-TB drives (leaving 2 spaces available for when prices drop later) would work like a charm.

Granted you could build nearly the same system with a standard beige-box and linux, what your money gets is a TINY form factor and the ease of not having to configure and administer something so heavily.

It should be noted however that my primary concern was data integrity and NAS, not storage space, for which I was willing to pay a premium. RAID is a no-brainer once you've set it up, but you should be very cautious about doing it in software (you start to lose some benefits). And unless you're RAIDing for redundancy, you drastically increase your chances of total data loss.

In your experience with NAS drives, are you able to do video editing from the drives over the network without speed limitations? I won't need to be streaming more than 1 video source at a time, but it might be HD.

Will I have any problems in iMovie if the content I am editing/importing/exporting is sitting on a NAS drive such as the Drobo or Netgear readyNAS?

This is the time I'd like to remind everybody that you don't need like drives in a Drobo. They can all be different. I currently have 2 - 500GB and 2 - 160GB hard drives for 760GB of redundant storage. When 1TB drives go down. I'll get one and simply pull out a 160, while the Drobo is on and doing its thing. And slide in a 1TB. Drobo will then manipulate the data, and then I'll have 1.1TB of protected storage, then 1.8TB when I replace the second 160.

I'm not anti RAID box, I just don't think its as "apples to apples" that people are assuming.

Very good point. Can you point me to some sources other than Drobo where you can buy hard drives compatible with their system? Or does it accept pretty much every type?

Also, same question to you regarding NAS speed. Is yours hooked up via wireless? Are you able to do video editing and such over the network speed connection?

no, no, no... I did not even take RAID into consideration. What I was saying is that with these devices you have a single connection to the PC (firewire) and the 1st drive connects to the 2nd drive, the 2nd drive to the 3rd, etc.
This is quite useful for iMac or laptop users who do not want to have a 2nd computer to serve files and also have a limited number of USB ports available.
-------

and now, for the most successful post ever made on these forums:

get a Windows Home Media Server
http://www.microsoft.com/windows/products/winfamily/windowshomeserver/buy.mspx

Ah... ok, I've got ya now. That DID sound a bit strange, albeit promising for my particular situation. I do exactly what you are talking about right now with my 2 current externals. I have a 750gb (for Time Machine use) hooked up via Firewire800, and a second 250GB with other files on it, daisychained through the Time Machine HD via Firewire 400. It works flawlessly, and lets me use my other Firewire drive for other things! This has to be one of the biggest benefits of Firewire!




Thanks for all of the help on this guys... I appreciate it!

 

[Wotan31]

In your experience with NAS drives, are you able to do video editing from the drives over the network without speed limitations? I won't need to be streaming more than 1 video source at a time, but it might be HD.

Are you kidding??? NAS volumes are simply *terrible* for anything that requires high sustained throughput or I/O - like video editing. Even SD video editing will be a disaster across a NAS volume and you can forget about HD.

You need something locally attached like a RAID0 firewire enclosure. Much faster and much cheaper than some kind of silly network storage server.

And don't even think about USB either. USB does not provide isochronous transfers like firewire does - critical to the real-time data transfer needed when editing video.

Don't forget to also buy another external drive to make backups. You'll crap your pants when your data drive dies and you realize you've never made a backup.

 

[fivepoint]

Are you kidding??? NAS volumes are simply *terrible* for anything that requires high sustained throughput or I/O - like video editing. Even SD video editing will be a disaster across a NAS volume and you can forget about HD.

You need something locally attached like a RAID0 firewire enclosure. Much faster and much cheaper than some kind of silly network storage server.

And don't even think about USB either. USB does not provide isochronous transfers like firewire does - critical to the real-time data transfer needed when editing video.

Don't forget to also buy another external drive to make backups. You'll crap your pants when your data drive dies and you realize you've never made a backup.

Well then! That answers THAT question! Thanks... and really, I guess I should have known that. I'm sure I could easily 'play' that video over the network (like my AppleTV does) but scrubbing, editing, etc. is completely different and way more I/O intensive.

Ahhh! This is getting way more complex than I thought it would be. I've got some serious thinking to do about what would be best for me right now, and for the next few years. It doesn't seem like you would suggest the Drobo either because it is only USB 2.0.

You mentioned the importance of RAID 0 for speed. How about RAID 5? It has the benefits of O (speed) and 1(redundancy), correct?

 

[jeremy.king]

Don't take this the wrong way but that sounds like a fubar situation waiting to happen, espicially since you are trying to do it to external drives.

Why do you think this? Is it your opinion, or do you have some intimate knowledge that this is less reliable/stable than a standalone RAID? I have had two externals in a mirrored RAID using Disk Utility and it's been fine for a long while now. Having two enclosures limits the chance of complete failure and makes me feel more comfortable than using one that could blow up catastrophically...

Although, when Drobo lowers their prices to something more reasonable - I may just buy one of those.