Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

I need more disk space - please help me figure out how to do it.

D

dknightd

macrumors 6502
Original poster
Mar 7, 2004
334
1
I need more disk space, I know this is a common problem, I've read many threads and discussions, both here and elsewhere. Hoping to get some input and discussion going on my particular situation. Please excuse the length of this post - skip to the end if you don't want details.

Current setup.
Mac pro 5,1 3.2 quad
tray 1: Stock 1TB disk (boot, apps, home directories)
tray 2: 2TB seagate ST32000542AS (1/3 for nightly boot clone - 2/3 for
occasional boot clone, and nightly home directory backup - keep 7 copies)
tray 3: 2TB seagate ST32000542AS Time machine of tray 1
tray 4: 1TB seagate ST31000340AS Various video (and other) projects

external - 4 2TB Lacie disks (these are actually 2 1tb drives in a single enclosure) 2 enclosures connect via esata (SeriTek/2ME4 pcie card), the other two connect via firewire. Two are used for data, the other two are for backup. Mostly music and music videos (Yes, I know 4TB is a lot, I've been collecting music for a long time!, and this is the family library: in case you are concerned this is not stolen content). I have another 2 of these disks on an offsite machine for backup.

My external disks are full, again. Time for more space. In the past I would have bought 3 more of the next larger size lacie disk (should be 4tb each by now). But lacie is not making this form factor any more it seems, and I've been using this shape of enclosure for so long that my workspace is kind of built around them). So, if I have to rebuild my physical work space maybe now is good time to consider other
options.

I don't want to have more than 2 active external media volumes. I would not mind consolidating them into one larger volume.

I've got most of my music on the computer now, and have slowed
down my purchases, so don't think this will increase much, but I've got grandkids now so expect that will generate pictures and videos. I'd like to add 2TB of space (plus 2x2 tb backups) to what I have now, and allow for room to grow.

I am a hobbiest. I'd hate to loose any of this data. But I've got three copies, so feel fairly safe. If I loose a day of work on this it is not the end of the world.
I back up important changes quickly. To loose all of it would be horrible, but I could still feed and cloth my family.

As I see it, I've got three basic options.
1) Something like the OWC Elite-AL pro (2x2tb in a single enclosure).
http://eshop.macsales.com/shop/Mercury-EliteAL-Pro-RAID Three of them.
This would basically extend what I had been doing, but I'd have to reconfigure
for the different shape. (or maybe just two of them, and combine some
of the leftover drives into a raid-0 or concatenated JBOD volume)
2) External hardware raid. Something like the OWC Qx2 (with 4x2TB disks - giving
me roughly 6TB of usable space)
http://eshop.macsales.com/shop/hard-drives/RAID/Desktop/
I could reconfigure my existing backup disks as osx spaned (contatinated) volumes for backup.

Here is my first question - sorry it took so long - I wanted to give you
some background.
Usually "enterprise" disks are recommended for parity based raid arrays.
As I understand it this is primarily because of the way they handle errors -
it is too easy for a desktop drive to be flagged as bad (even though
it may not be) triggering a raid rebuild - if a second disk happens
to incorrectly be flagged as bad - everything is lost.
But OWC sells both an "enterprise" version and a version using a desktop drives.
Why, if only "enterprise" drives are suitable for parity raid arrays?

3) A larger external box. Something like the SANS DIGITAL TowerRAID TR8M-BP
http://www.newegg.com/Product/Product.aspx?Item=N82E16816111141
It comes with an pci-esata card. But I understand the Highpoint card/software
is not held in high regard. Again for raid-5 (parity raid) "enterprise"
drives are highly recommended. Too bad they cost much more than similar
"desktop" drives (WD black comes to mind)

I'm tempted to do 3). Run the SAN box as either raid 5, or more likely just
a JBOD concatenated (spanned) volume. I know both are risky with "desktop" drives
and a combination hardware/software raid. Then make two backup sets (one local
and one remote as now) using
osx builtin ability to span disks using my existing disks.

Remember, this is a hobby. I'd like to keep my costs as low as possible.
But I don't want to do something that is stupid and I will regret.

So, please, tell me how you would turn 4TB of disks (with two backups)
into 6TB of disks (with perhaps room to expand again without
redoing everything - and again with two backups)

Thanks
 
philipma1957 said:
be patient and wait for me to test this card
Click to expand...

I can be patient. Seems like the ts2ct is a 2 disk raid0 (or raid1) box, similar to the OWC Elite-AL pro.
If this is the direction I decide to go, I've give it a close look, not sure how
I would choose one over the other. But I'm not there yet.
I don't think I'd need another esata card - I've got two ports unused
on my current one.
 
Last edited by a moderator:
dknightd said:
snip...
tray 1: Stock 1GB disk (boot, apps, home directories)
tray 2: 2GB seagate ST32000542AS (1/3 for nightly boot clone - 2/3 for
occasional boot clone, and nightly home directory backup - keep 7 copies)
tray 3: 2GB seagate ST32000542AS Time machine of tray 1
tray 4: 1GB seagate ST31000340AS Various video (and other) projects

external - 4 2GB Lacie disks
...snip
Click to expand...

Might want to change those to TBs.
 
Vader said:
Might want to change those to TBs.
Click to expand...

Silly me - user error - the most common type of error. I think I've changed
all gb to tb - if not, and I refer to disk space, assume TB.

How embarrassing.

So, how would you turn 4TB of disk into 6TB of disk? And still be reasonably well backed up, and as cheap as reasonable?
 
Just to throw a possibly crazy idea out there.... have you thought about adding an ethernet attached storage device? Either with an ethernet port on the enclosure, or to an old Mac Mini with FW800?

For backups, you don't need the fastest connections. Ethernet allows you put the drives further away than your current set up - in a closet on the other side of the wall, perhaps. You might be able to repurpose the enclosure you are reluctant to lose, and save yourself the trouble of rejigging your work space.

Just a crazy thought.
 
dknightd said:
I need more disk space, I know this is a common problem
....

My external disks are full, again. Time for more space. In the past I would have bought 3 more of the next larger size lacie disk (should be 4tb each by now). But lacie is not making this form factor any more it seems, and I've been using this shape of enclosure for so long that my workspace is kind of built around them). So, if I have to rebuild my physical work space maybe now is good time to consider other
options.

I don't want to have more than 2 active external media volumes. I would not mind consolidating them into one larger volume.

...

As I see it, I've got three basic options.
....
Click to expand...

No you have four options. At its core you don't have a space problem you have a "pack rat" problem. What you are doing is gathering more and more and more stuff and never deleting anything.

The four option is to find the stuff you don't look at very often ( Haven't touched in 2-3 years ) and move to a set of nonactive archives. You'd only really need max two copies for disks that are powered down most of the time. You need to spin them up a couple times a year perhaps to check it was OK but for most part just quietly store them.



If you don't control what you are keeping spun up on active drives then eventually you end up with either a huge storage budget (much larger than it needs to be) or setting your self up to backup problems ( because gets to big to practically backup. For example let the time slide because most of the data on the volume is static so ..... ).

If your data collection growth rate is outstripping the new drive sizes ... then seriously need to consider whether all that stuff really needs to be online.
Making digital copies and keeping them in the house is one thing. Keeping them spun up for can get instant access is quite another. If there is no business reason to keep stuff spun up, then the simplest solution is don't.

There is a unix utility called "find". You can issue commands to find that looks at the "last accessed" and "last modified" times on your files.

find /ArchiveVolume not -neweraa SomeYearOldFile -print

That will produce a list of files not accessed since that 'SomeYearOldFile' has been accessed . ( without the 'not' it would be the newer stuff; can find changes to the file system too. )

With only slightly fancier scripting you can store that list. Then have another script copy those files to another volume (rsync run as root can copy them over with all the current metadata properties. )

None of the storage or system vendors really want to address this because it wide spread pack rat behaviour sells more systems. People buy bigger computer because disk is full and computer kind of old. And others buy storage towers and fancy cards to connect them because spinning disks and not really accessing the data.
 
deconstruct60 said:
No you have four options. At its core you don't have a space problem you have a "pack rat" problem. What you are doing is gathering more and more and more stuff and never deleting anything.

The four option is to find the stuff you don't look at very often ( Haven't touched in 2-3 years ) and move to a set of nonactive archives. You'd only really need max two copies for disks that are powered down most of the time. You need to spin them up a couple times a year perhaps to check it was OK but for most part just quietly store them. ......
Click to expand...

this is what I was driving at. I have decided that much of my tv recordings can sit on unused hdds in a case like this.

I have one hdd with a lot of Saturday nite live one hdd with jay leno and conan obrien they don't need to be online. you can pop them in when you want them.

I only mention the card i purchased because it may be bootable with the above unit. this is a truly low cost method for you. 5 rubber cases a newer tech unit and you say you have your own esata pcie card installed. so it is 35 for the cases 35 for the newer tech 70 and 110 bucks for each 2tb drive

this is low cost you get esata speed and if my card i purchased boots you can toss an osx on the drive along with the movies and tv shows.
 
Last edited by a moderator:
My current thinking is:

the SANS DIGITAL TowerRAID TR8MP

http://www.newegg.com/Product/Product.aspx?Item=N82E16816111140&Tpk=TR8M-P

I'd like to use "enterprise" drives, but think instead I'll use constant rpm (7200) drives,

either HITACHI Deskstar H3IK20003272SP
http://www.newegg.com/Product/Product.aspx?Item=N82E16822145369

or

WD black
http://www.newegg.com/Product/Product.aspx?Item=N82E16822136456

in a JBOD span

(I don't think I trust the highpoint card for raid 5 - so no real point in using enterprise drive - and I'm really not up to spending $500 for proper hardware raid controller)

I'm still open to suggestions. I have up to $1k allocated for this expansion.

I still need to confirm I can move a macosx concatenated drive from one machine to another and have it work.

Comments welcome
 
You seem very concerned about backups, so I presume that you heavily value the data that is stored on this (very large) amount of storage. So, please allow me to ask...do you keep off-site backups anywhere? You back things up multiple times, but it appears from your post you keep all of those drives in the same place. This deals well with hardware failure on one/more of those drives, but a fire or other event in the location your system is at would destroy all of the data and all of the backups.
 
I have two (or more) backups of everything I think is important.

I use ccc and silverkeeper to make local backups.
rsync for remote backups (which I use, so I have
all my music in the remote location)

My thinking is, One local backup, and one off site backup, is the minimal for anything you really care about.
 
You need to simplify! Your current system is fragmented, and subject to data loss by being too closely tied together. You have a source drive and two backup drives inside the same computer, and the rest of them electrically tied to that computer. One bad power surge, and you lose everything!

I would second the suggestion of a Network Attached Storage device. Connected via Gigabit Ethernet, this is faster than even FW800, and fast enough for any but the fastest of today's mechanical drives. If you are going to RAID 5 (or similar), the bottleneck will be the RAID array, not the connection.

In your case, with as much data as you have, I would suggest something like a Drobo or Windows Home Server. These devices allow you to have a number of drives, even of different sizes in your storage 'pool' (as opposed to RAID 5 which requires all drives to be the same size), and to dynamically upgrade the storage space by either adding new drives or by replacing the smallest drive with a larger drive as drive capacities increase and prices come down. Most devices like this also include usb or esata connectors, so you can expand beyond the confines of available bays, if necessary - but while keeping only one storage pool.

Having the device on the network also means that any computer on your LAN can access it - wireless or wired - without having to leave a host workstation running. It also means it can be placed in a basement or closet, rather than cluttering up your desk. Drobo does make devices that are esata/usb/fw connected rather than Ethernet - but I would encourage you to go the network connected route - it is much simpler in the long run.

Lastly, I agree that you should add some off-site storage - perhaps through the cloud. A service like Crashplan allows you to store an unlimited amount of data on their servers for a nominal yearly fee. The first backup will take a very long time, but after that, it only sends block level file changes, which means backups can occur quickly and in the background, every day. In my case, I only back up really important data to the cloud - everything else is duplicated once at home.
 
Thanks for your concerns about my backups. To clarify, I have one (or more) backup of everything a care about off site, and one (or more) copy locally. Yes it would be better if the local backups were not in the same room, on the same computer. Maybe.
My experience has been that human mistakes are more common than hardware failures, or fires, so I guess I'm OK with my arrangement. YMMV
But I'll take your suggestions to heart - I've got enough macs littered
around the house I could easily hang my local backups off a different
mac in a different part of the house, if there was an advantage to doing that
(and there might be) and it was easy to use.

For right now I'm seeking thoughts on my media storage needs.

I'm looking for advise on how to store 6-8 TB of media (photos, music, videos). With at least a semblance of backup capability. I have 4TB now,
expect growth of about 1TB/year, would like 3 years of growth capability.

Lots of posts about what I'm trying to do is stupid, and maybe it is, not much on how to deal with what I want to do.

So, how would people suggest I add 6-8 TB of storage to my computer, for less than $1k? (I've got enough leftover disks that I can make two backups - one local, one remote, both using just osx spanning)
 
um, several of us suggested network attached storage - in my post above you, I suggested WHS or a Drobo specifically because of the ability to dynamically add space. It'd be better than a 3 year system, because you can continue to replace the smallest disk, one at a time, as disk prices come down, without even having to take the storage pool off-line. The Drobo is a bit more expensive, but a little simpler than WHS. WHS can be bought cheap, or even built yourself from spare parts.

These systems are more flexible than standard RAID 5 systems. I use RAID 5 for my storage at home - but I built it 5 years ago. There is no way I would use a regular RAID 5 array today.
 
OK. thanks. I'll look into the drobo again. If it can deliver gigabit ethernet through put, and decent reliability, it could suit my needs just fine.
 
One of the things about any RAID 5 system is that it is not going to be particularly fast compared to a single disk - Gigabit, FireWire, eSata, whatever will all be faster than a typical RAID 5 array can physically read or write the data (reads are much faster than writes). Neither the Drobo or WHS will be particularly fast, either. You don't use this type of storage system for speed - you use it for reliability, convenience, and flexibility.
 
what i've done with my home system is this:
1: 3tb WD green (boot drive)
2: 2tb WD black (work drive)
3: 2tb WD green (raid stripe)
4: 2tb WD green (raid stripe)

4tb raid volume for time machine

makes my backup volume more unreliable, but that's a risk i am willing to accept for the convenience of keeping everything in the machine and doesn't rely on external hardware. it's for time machine only.

in the future, i will get a 3tb black drive for boot and have a 6tb (2x3tb) for time machine
 
I decided to go with TR5MP enclosure, with 3x2tb hitachi disks.
I'll run it as an osx spanned volume. That will make it easy to grow.

I'll convert my existing external drives into two 3x2tb osx spanned
volumes (one for off site, the other I'll hang off an old g5)
for backup.

Not an ideal solution, but should serve my purposes just fine.
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back