Q: JBOD, Spanning, Striping, And It's Usage For TM Backups

[JulianBoolean]

[..]

QUOTE: "JBOD, meaning "Just a Box Of Disks", "Just a Bunch Of Drives", or, as a recursive acronym, "JBOD's a Bunch Of Disks"[citation needed], is used to refer to two distinct concepts: 1. all disks being independently addressed, with no collective properties – each physical disk, with all the logical partitions each may contain, being mapped to a different logical volume: just a bunch of disks. 2.concatenation, where all the physical disks are concatenated and presented as a single disk. <snip> In that it consists of an array of independent disks, it can be thought of as a distant relative of RAID. Concatenation is sometimes used to turn several odd-sized drives into one larger useful drive, which cannot be done with RAID 0. For example, one could combine 3 GB, 15 GB, 5.5 GB, and 12 GB drives into a logical drive at 35.5 GB, which is often more useful than the individual drives separately.

Concatenation or spanning of disks is not one of the numbered RAID levels, but it is a popular method for combining multiple physical disk drives into a single virtual disk. It provides no data redundancy. As the name implies, disks are merely concatenated together, end to beginning, so they appear to be a single large disk. It may be referred to as SPAN or BIG (meaning just the words "span" or "big", not as acronyms).

The Mac OS X 10.4 implementation – called a "Concatenated Disk Set" – does not leave the user with any usable data on the remaining drives if one drive fails in a concatenated disk set, although the disks otherwise operate as described above." END QUOTE. Source: Wikipedia, Non-Raid Drive Architectures

[..]

Consider ....

Two Identical 1500GB External FW Drives Concatenated (Spanned) together for TM backups =
• 3000GB Volume
• Data Loss if any drive fails
• Speed of 1 drive

Two identical 1500GB External FW Drives Striped for TM Backups =
• 3000GB Volume
• Data Loss if any drive fails
• 2x speed of 1 drive (roughly, I know does not scale perfectly)

Then...

Q: I might as well 2x R0 stripe my external FW drives? While I'm not super concerned about how fast my back ups are writing, I'd be happy to have them happen faster, as it would shorten the time when there is background stuff happening whle I am working. I see the same downside (loss of data in both cases) but an obvious upside to striping in this particular case. Am I missing something here? Over thinking this? Thoughts?

- Julian

 

[Transporteur]

Just for the record, a single 1.5TB drive can already saturate the FW800 bus, so speed shouldn't be of concern in this scenario at all.

 

[dknightd]

pros: raid 0 is faster
spanning disks allows you to grow the volume without a reformat.

Backing up is a three lane street - your source disk has to be fast, the software has to be fast, and the backup disk has to be fast (and if you backup across ethernet/internet that has to be fast as well). The slowest link will determine the speed of the whole process.

If you do not expect the size of your backup to change, and you need more space than one disk will hold, then raid0 is a good choice if you can use its speed. If you think you might need to have more space, and a spanned disk will keep up, then it might be a better alternative.

There is no right answer.

 

[dknightd]

Just for the record, a single 1.5TB drive can already saturate the FW800 bus, so speed shouldn't be of concern in this scenario at all.

good point. but if there are two FW buses, then what?

 

[Transporteur]

good point. but if there are two FW buses, then what?

Fair enough.
Time to find out how many independent buses the MP actually has. Considering the 4 ports, it might well be two buses.

 

[JulianBoolean]

Fair enough.
Time to find out how many independent buses the MP actually has. Considering the 4 ports, it might well be two buses.

Hmm... lemme take a look. I have them both hooking into the two available FW ports on back of the mac, independently, via two FW 800 cables (not daisy chained together). As for the two FW ports on the front, I typically use one of those for my portable FW "shuttle" that I use to keep all my music and my take home work. The other FW port one on the front is typically not used. Is this the info you need?

 

[Transporteur]

Is this the info you need?

Unfortunately, no. The discussion is more concerned with the actual amount of FW buses the Mac Pro has.

Let me explain it like this: You've got two discs attached to separate FW ports, which would mean that each disc can be accessed with the full FW800 speed (which is something around 75 to 80MB/s). This would, however, imply that each port has its own bus. In case that the two ports share a single bus, the overall performance would be limited to the mentioned 80MB/s, instead of allowing each drive to be accessed with that speed.

Since I can't find any information about how many buses the MP has, you could do a quick performance test which should clear things up (or we could just wait for nanofrog ).
Attach both FW drives and copy data to or from them from you internal drives. Apparently you'd have to use either separate internal drives, or an array that is fast enough for supplying both external drives.

 

[philipma1957]

a) the 2010 base 2.8 I own has 1 fw800 bus.


b) depending on your enclosure. the owc qx2 if you have it you may as well not use any of its jbod features. reason is the disks are not separated and if 1 fails the entire jbod will fail.

c)now if you have the rosewill 8 bay or the sans digital 8 bay and run jbod each disks stands alone.

so JulianB what is your enclosure ?


the thumbnail show the one and only bus then a built-in hub i have

the owc qx2 a sans digital and a seagate fw800 on one port a 3 long d-chain

on the other port i have a sans digital . the mac pro has a lot of limits with its fw800 setup once you involve 2 or more fw800 units you can drop speeds. been there done that.

 

[khollister]

a) the 2010 base 2.8 I own has 1 fw800 bus.


b) depending on your enclosure. the owc qx2 if you have it you may as well not use any of its jbod features. reason is the disks are not separated and if 1 fails the entire jbod will fail.

c)now if you have the rosewill 8 bay or the sans digital 8 bay and run jbod each disks stands alone.

so JulianB what is your enclosure ?


the thumbnail show the one and only bus then a built-in hub i have

the owc qx2 a sans digital and a seagate fw800 on one port a 3 long d-chain

on the other port i have a sans digital . the mac pro has a lot of limits with its fw800 setup once you involve 2 or more fw800 units you can drop speeds. been there done that.

Uh, why are you using FW on a Mac Pro? you can get a 2 port eSATA card for $49 or one for $79 that supports port multiplication and RAID. Use eSATA. If you ever need to boot from one of them, hook up a firewire cable.

There are PCIe slots in that thing for a reason

And using stripe sets on FW is rather silly - a single drive can easily hit 150 MB/s on an a SATA connection. The same drive in most FW enclosures is going to get 65-80 MB/s via FW 800. FW isn't even close to bandwidth available from current desktop drives, what's the point of RAID 0?

 

[khollister]

Everybody tries to make this so hard. Time Machine is primarily valuable for stuff that is either fairly volatile (as in you are modifying or adding new files on a regular basis) or susceptible to damage from either software installs/bugs or operator error. It is also not a disaster recovery solution - it is connected to the computer at all times and isn't directly bootable or even useable without TM.

I find it really unlikely that anyone actually has > 2TB of active files. Most stuff is either archival in nature or is rarely touched other than to read it. That type of stuff can be periodically cloned via CCC or SuperDuper and stored elsewhere for true disaster recovery purposes. The active part of your filesystem should also be clones and stored offsite on a regular schedule. TM is handy for hourly archives in case a drive fails or you nuke something by accident in the part of your filesystem that you are actually writing to on a daily/hourly basis.

Therefore, the need for large RAID or JBOD arrays for TM is "making it too hard" IMHO. Use a 2TB TM drive of "works in progress" and get a bare drive dock or two and a fleet of bare drives to make 2 or more rotating sets of cloned backups (at least one offsite in case of fire or theft). You can set up CCC to automatically do the clone overnight - next day pop in a new drive for the next scheduled clone. Unlike TM, you can break up the clone scripts to spread the backup over multiple drives.

Time Machine is handy, but it has really lulled people into thinking it is the final word in backup. It is a great tool, but it's only part of the solution.

Often the thing that trips people up is large iTunes libraries where they are adding new music or video regularly. I have always solved that problem (mine os almost 500 GB) by putting the iTunes library on a RAID 1 mirror and forgetting about TM for it. Since high bandwidth is not necessary to stream music or video, I currently have this in a very small OWC RAID 1 enclosure for 2 2.5" drives that is bus powered from the FW 800 connection. It just sits on the top of the MP with a single FW cable connection, and I clone it every couple of weeks in case the entire device blows up or the house burns down. The worst that happens is I lose one or two TV shows I bought or a few songs, but I am continuously covered in case of a single drive failing

 

[JulianBoolean]

Everybody tries to make this so hard.

Nobody is making anything hard. I'm asking a simple question in a very precise manner. The last time I asked a related question, I was a bit sloppy with my verbiage, and the whole thread got off on the wrong foot. I'm determined not to repeat.

Time Machine is primarily valuable for stuff that is either fairly volatile (as in you are modifying or adding new files on a regular basis) or susceptible to damage from either software installs/bugs or operator error. It is also not a disaster recovery solution - it is connected to the computer at all times and isn't directly bootable or even useable without TM.

I understand what you are saying, but know not why you are saying it. The implication here being there is some misunderstanding of the basic premise of Time Machine?

I find it really unlikely that anyone actually has > 2TB of active files. Most stuff is either archival in nature or is rarely touched other than to read it. That type of stuff can be periodically cloned via CCC or SuperDuper and stored elsewhere for true disaster recovery purposes. The active part of your filesystem should also be clones and stored offsite on a regular schedule. TM is handy for hourly archives in case a drive fails or you nuke something by accident in the part of your filesystem that you are actually writing to on a daily/hourly basis.

Again, it seems you presume there is a misunderstanding of the proper elements of a back up routine? I've left out the fact that I do nightly updates of my boot and apps drive, VIA super duper, because it seems a minor detail to the central question.

Therefore, the need for large RAID or JBOD arrays for TM is "making it too hard" IMHO. Use a 2TB TM drive....

Aha. Here is the nub of the issue, and perhaps goes a long way in what seems to be a misunderstanding, or mis-interpretation. I don't have a 2TB drive. I bought the two 1.5TB drives several years ago. A single 1500GB drive simply isn't big enough, because as I'm sure you are aware, it's best to have double the space needed so that time machine can save it's hourly/daily/weekly iterations. I'm concerned with getting the most out of what I already have for reasons of economy.

- Julian

 

[khollister]

Julian

Your question was why not stripe if the OS X spanning implementation also had a single point of total failure. I can't disagree with the assertion if that is the only two options. However both of those options result in significantly degraded reliability of a subsystem that is supposed to increase overall system reliability.

My comments were intended to point out that instead of focusing on choosing from a lose-lose scenario, you might solve your fundamental problem in another way that gets you to where you really need to be - increasing system reliability.

The issue is that TM can only use a single volume and you do not have a inexpensive way to create a larger volume without compromising MTBF. I was proposing a different form of solution - find a way to maintain or increase reliability while reducing the volume capacity required for TM.

The basic idea is to only use TM for data that changes more frequently than the rate at which you perform snapshots with SuperDuper, CCC or whatever.

People look at TM as there primary disaster recovery solution and thus believe everything has to be covered by TM. My point was to view TM differently and realize it is a poor substitute for proper offsite backups. If everything is backed up via drive clone sets, then TM is only needed for parts of the filesystem that change between backups, thus eliminating the use of risky RAID 0 volumes for the incremental archives of TM.

If I understand you right, you are making daily clones of your OS and application directories, but are relying primarily on TM for the stuff that actually cannot be easily recreated/installed. Since TM relies on the TM volume being continuously connected to the computer, it only offers protection in the case of a drive failure or operator/software mistake. Not that those aren't important, but the ultimate "disaster" scenario is theft/fire/flood/direct lightning strike that eliminates the primary drives as well as the TM volume that is collocated (unless you are doing this over a network to a remote location - slow).

Just saying ...

 

[JulianBoolean]

Unfortunately, no. The discussion is more concerned with the actual amount of FW buses the Mac Pro has.

Let me explain it like this: You've got two discs attached to separate FW ports, which would mean that each disc can be accessed with the full FW800 speed (which is something around 75 to 80MB/s). This would, however, imply that each port has its own bus. In case that the two ports share a single bus, the overall performance would be limited to the mentioned 80MB/s, instead of allowing each drive to be accessed with that speed.

Since I can't find any information about how many buses the MP has, you could do a quick performance test which should clear things up (or we could just wait for nanofrog ).
Attach both FW drives and copy data to or from them from you internal drives. Apparently you'd have to use either separate internal drives, or an array that is fast enough for supplying both external drives.

Transporteur,

Awesome info, thanks! I did not consider the inherent slowness of a FW connection. I currently have both drives spanned together and am getting reads at 83.5 and writes at 66.2MB/s. See attached for details. Don't really know from this result whether there are two buses?
Thanks!

KHolister,

I appreciate you thinking about the issue holistically, your point about also keeping stuff off site as insurance for fire, theft, etc is a good one.
Thanks!

PhilipMa,

There isn't an enclosure housing these two FW drives together. Here's what I have.
Thanks!

Julian

 

[philipma1957]

those drives are maxed for speed.

fw800 on a mac pro will not exceed what you are doing.

they are bootable that is nice but for a tm not important. the photo in your link lacks info on your enclosures do they have esata? do they have usb2? do they have a power supply? by the way did you buy them empty or with drives.

always buy kit versions from owc. they will void your warranty if you buy a drive version and pull the drive out.

 

[JulianBoolean]

PhillipMa -

I have 2x the kit version. They shipped with a disk already inside. Each has a dc power switch, and esata port, a usb port, a fw 800 and a fw 400. The screen grab shows the back and all the component parts I have.

 

[Inconsequential]

Span doesn't loose the data.

It just fills one disk up and then starts writing to the other.

This is why I've bought the OWC Dual Bay things for my backups.

 

[khollister]

Span doesn't loose the data.

It just fills one disk up and then starts writing to the other.

This is why I've bought the OWC Dual Bay things for my backups.

That would solve Julian's problem if the OWC enclosure works that way. I still think folks are spending a lot of intellectual & emotional energy trying to create large volumes for TM when it really isn't buying them much from an overall backup strategy.

I did have one of the OWC dual enclosures a few years ago (IDE ATA days) and it had a pretty noisy fan. The Guardian Maximus (RAID 1) box is much quieter, so hopefully the new ones are better.

 

[Inconsequential]

That would solve Julian's problem if the OWC enclosure works that way. I still think folks are spending a lot of intellectual & emotional energy trying to create large volumes for TM when it really isn't buying them much from an overall backup strategy.

I did have one of the OWC dual enclosures a few years ago (IDE ATA days) and it had a pretty noisy fan. The Guardian Maximus (RAID 1) box is much quieter, so hopefully the new ones are better.

They are still fairly noisy tbh, but the fans turn off after 2 mins of the MP going to sleep so it doesn't bother me.

And I haven't tested the file recovery method, I just hope when I come to add a second 2Tb drive to the enclosure it works that way

 

[rtrt]

a

 

[JulianBoolean]

That would solve Julian's problem if the OWC enclosure works that way. I still think folks are spending a lot of intellectual & emotional energy trying to create large volumes for TM when it really isn't buying them much from an overall backup strategy.

OY. Dude. Again with the inaccurate assumptions. Here's the deal. I create illustrations, CGI, and photo retouching for large volume print advertising. The images are often 8GB as they sit on the disk. I save every 20 minutes. I often work on 1 image for several weeks. After the image is approved, then I need to roll out 8-10 versions at various sizes and color profiles for specific press runs. I also need to keep all my incremental save iterations on hand because Art Directors have a habit of changing direction. I also need to keep all of the jobs I did for the last several years within easy reach because I'm using and mixing layers, wireframes, and render surfaces from previous jobs. So yes, I abso-freakin-lutely need 3GB for Time Machine. No, I don't have a 3 GB drive. That is whole point of spanning. Yes, I do nightly backups of of my boot and Apps drive. No I don't save anything off site, and I should. I appreciate your oft stated viewpoint here, but if you've still got a burr in you saddle about "folks" (who ever that is) leaning to heavily on Time Machine, and overlooking a larger plan regarding disaster recovery perhaps you might start your own thread about it, as that conversation kinda falls outside the scope of what I need help with.

 

[JulianBoolean]

Span doesn't loose the data.

It just fills one disk up and then starts writing to the other.

This is why I've bought the OWC Dual Bay things for my backups.

Hmm... here's where verbiage really get's important. Concatenated (spanned) disks created by the OS disk utility, as I understand it, (and my understanding is really thin and based upon a single line in the opening wikipedia quote) will suffer total data loss when any one member of the sets dies. That's kinda why I'm posting here, as I always take wikiland info with a bag of salt. Do you have any other sources or personal experience that would help make this issue more certain for me?

And yes, regarding your comment about striping increasing the speed, that is something I was considering, given that concatenated (spanned) drives and striped drives will both result in total data loss should any single member fail. It got me thinking about upside and downsides, thus my line of questioning here. Turns out, I over looked the slowness of FW and the number of available buses inside a mac for FW is still uncertain.



Thanks!

Julian

 

[dknightd]

JulianBoolean,
A data recovery program is more likely to be able to retrieve usable data from one half of a spanned array, than it is from one half of a striped array.
But I would not worry about that. For all practical purposes I think you are correct to assume that the loss of one disk will result in the loss of your backup.
Since you are concerned about backup speed, and your external drives both support esata, I'd consider getting an PCI esata card and setting your two disks up raid 0. You'd have to accept a larger probability of loosing that backup, but, you have others so are reasonably well protected.

 

[Inconsequential]

Hmm... here's where verbiage really get's important. Concatenated (spanned) disks created by the OS disk utility, as I understand it, (and my understanding is really thin and based upon a single line in the opening wikipedia quote) will suffer total data loss when any one member of the sets dies. That's kinda why I'm posting here, as I always take wikiland info with a bag of salt. Do you have any other sources or personal experience that would help make this issue more certain for me?

And yes, regarding your comment about striping increasing the speed, that is something I was considering, given that concatenated (spanned) drives and striped drives will both result in total data loss should any single member fail. It got me thinking about upside and downsides, thus my line of questioning here. Turns out, I over looked the slowness of FW and the number of available buses inside a mac for FW is still uncertain.



Thanks!

Julian

The OWC guy I was talking to said that the data will still be available if one disk fails.

However I have a secondary backup anyway so even with two disk failures I still have one copy

 

[philipma1957]

..... Turns out, I over looked the slowness of FW and the number of available buses inside a mac for FW is still uncertain.



Thanks!

Julian

Julian mp has 1 fw800 bus.

If you want I can cook up the best way for you to have a nice tm 4tb. very simple a pair of wd2003fyys 2tb enterprise drives in raid0 use them in a sans digital towerstor2 hook up via esata
below.

the drives are 500 the sans digital is 180 the esata card is 16. cost is 700 but your speed will be around 115MB/s and that esata card boots!


if booting is not needed you can get faster esata card like owc's for 50 bucks it will not boot but your speed will be about 170MB/s.

If you buy the faster card and have to boot you can hook the gear up to a fw800 port. I push this gear because it is stable and seems suited to your needs

 

[nanofrog]

JulianBoolean, what happens is that if a disk fails in a JBOD, the data is still there, but the system can't read it as one of the members of the set is dead.

What this means, is you have to get the data off via a data recovery tool such as Disk Warrior. But it's still there on the undamaged drives.

This is not possible if the data is stored on a stripe set however, (all data = gone in the event of a disk failure given how it's spread across all members in small pieces), so must be restored from backups when a failure occurs. Which is why a stripe set isn't so wonderful as a backup/archival location (those that do use it are betting on the fact that the data on the primary location won't fail at the same time the backup location failed, but tend not to take into account if the disks are the same model with sequential serial numbers <or nearly so; remember, you don't want disks in a set from the same batch if at all possible due to an increased failure rate if there is a manufacturing defect>).