SAS drives? Firewire storage?

[geoffreak]

If I decided to end up going for a Mac Pro with pure speed, I would want to fill it with 4 SAS drives. I've heard that SAS drives are completely unrivaled in speed (15,000 rpm ), but does this really correlate to high speeds? Is Apple's RAID card the best solution for SAS drives? I would likely use RAID 5 setup for striping benefits as well as data redundancy.

Now, I'm a bit of a storage junkie, and SAS drives won't be a viable storage solution for everything I have already (and even more in the future), so external storage would be a must. I have a drobo already, and it is a much cheaper solution than getting something like an Xserve RAID, and firewire 800 is plenty fast for streaming video, even to multiple sources. My question is what will happen if I hook up 4, 5, or even 6 drobos? How does daisy-chaining many storage solutions affect speed? I won't be buying a Mac Pro until the next generation, and I'm assuming that there will be 2 firewire 800 ports, but will this even be much of an advantage? How about RAID striping across chained firewire?

Hope all these questions can be answered.

 

[sparkie7]

If I decided to end up going for a Mac Pro with pure speed, I would want to fill it with 4 SAS drives..

why dont you fill it with 4 SSDs in a RAID setup instead

 

[geoffreak]

Correct me if I'm wrong, but I think SAS drives offer better performances, higher storage, and cheaper prices.

 

[sparkie7]

Correct me if I'm wrong, but I think SAS drives offer better performances, higher storage, and cheaper prices.

Better performance - No (if u install a Fast SSD)
Higher Storage - Yes
Cheaper - Definitely yes

 

[geoffreak]

Well, I'm not making a light-speed rig, so I think that justifies me choosing SAS for their better price/speed ratio.

 

[nanofrog]

If I decided to end up going for a Mac Pro with pure speed, I would want to fill it with 4 SAS drives. I've heard that SAS drives are completely unrivaled in speed (15,000 rpm ), but does this really correlate to high speeds? Is Apple's RAID card the best solution for SAS drives? I would likely use RAID 5 setup for striping benefits as well as data redundancy.

Generally speaking, they are fast. But you have to keep in mind the usage (workstation or server) to choose the best for your needs. For example, Fujitsu makes wonderful workstation drives, due to exceptional random access speeds, while the Seagate Cheetah 15K.6 drives, while technically the fastest SAS drives out there, are better at server applications. (Large file transfers).

You might want to consider the Velociraptor 300GB drives, as they are also very quick. Like SAS, it's more expensive per gigabyte, and the capacities are smaller.

If you need capacity, there are SAS drives available, but they are slower. Cost wise, and the fact you are using RAID to improve speed, you might be better served with SATA drives. (Keep in mind, I'm referring to enterprise drives, not consumer, as they really aren't up to the task of RAID, especially where reliability is concerned).

I presume you're aware of the issues with RAID 5 (write hole issue). If not, there's a description of it in Wiki. Definitely worth reading. If you do go this route, just make sure to keep the array size down. A max of 4 drives will help with this.

Now, I'm a bit of a storage junkie, and SAS drives won't be a viable storage solution for everything I have already (and even more in the future), so external storage would be a must. I have a drobo already, and it is a much cheaper solution than getting something like an Xserve RAID, and firewire 800 is plenty fast for streaming video, even to multiple sources. My question is what will happen if I hook up 4, 5, or even 6 drobos? How does daisy-chaining many storage solutions affect speed? I won't be buying a Mac Pro until the next generation, and I'm assuming that there will be 2 firewire 800 ports, but will this even be much of an advantage? How about RAID striping across chained firewire?

Hope all these questions can be answered.

Daisy chaining will slow you down, as the bandwidth is shared. Internally, the drives are switched in the Drobo, so only one drive would be read/written at a time.

Another possibility would be to use a RAID card that has the ability to run both internal and external drives. You can even use an internal only, and use a special cable, (I can provide info where to get them if needed), to run from the internal port(s) out an open PCIe slot (cover), and run the other end to an external enclosure. Just don't span an array between internal and external drives. Way more hassle than it's worth (recovery issues, particularly in regards to power faults).

Areca makes some really good SAS cards, and are hands down better than Apple's offering. I'd recommend looking seriously at the ARC-1680 series. Great features, particularly the backups of the Partition Tables. You can also upgrade the memory on it. It boots. Many more.

They also have an external 4 port MiniSAS connector (SFF-8088). This cable is easy to find. You can attach 4 additional drives to an external enclosure. Any more, and you need a SAS expander, and those aren't cheap.

why dont you fill it with 4 SSDs in a RAID setup instead

SSD's aren't exactly the most mature products yet. Capacities are very small, and they're still quite expensive.

Personally, I still have some concerns with the reliability. MLC and SLC just don't have adequate write cycles yet, IMO. Even with write leveling, and a couple of other tricks to aid in extending the useful life of the drives.

Further, current OS's are still optimized for mechanical drives. This is problematic, as it can fragment, and even prematurely kill an SSD.

This will change, but I think it's worth waiting for more data, and possibly some advances yet to be seen. FeRAM is one of them. 1E16 write cycles vs. 10k or 100K of the current Flash technologies.

 

[nanofrog]

Better performance - No (if u install a Fast SSD)
Higher Storage - Yes
Cheaper - Definitely yes

Particularly in the case of Intel's X-25 series, absolutely.

For RAID, SAS is a better choice ATM. Stable, and has a proven track record for both their speed and more importantly, reliability.

 

[geoffreak]

My primary use for these drives will be in a workstation, so higher random access times are preferred, which is one of the reasons I want an SAS array. The SAS array will only be running my applications along with offering some internal storage.

WD's Velociraptors don't quite cut the quality I desire because they don't offer enough internal storage and they are more expensive than some SAS drives that are faster and offer more space.

Let me clear something up: I am planning to RAID 5 the internal SAS drives alone. The external drobo setup will be completely separate. Each drobo also has only four drives in a special "beyond-RAID". This will ensure that the write hole issue doesn't exist in my setup. If I recall correctly, there is a RAID card for the internal drives that offers battery backup to avoid the cause of the issue.

Regarding the slowing down of daisy chaining, I don't think I would want to daisy chain any more than 2 of the devices. If the future Mac Pro does indeed have 2x firewire 800 ports, this would only allow for up to 4 drobos, which won't be enough for what I will end up storing. Now the question becomes if I can use a PCI express 2.0 card to add more firewire 800 ports. I plan on having a RAID card and two graphics cards (not doublewide), but Apple's website allows for 4x ATI cards in the current model, so I'm assuming there be at least this many in the future. With that note, it boils down to if the firewire cards intended for PCs will work with Macs, as I don't think there is a Mac only firewire PCI card.

 

[nanofrog]

My primary use for these drives will be in a workstation, so higher random access times are preferred, which is one of the reasons I want an SAS array. The SAS array will only be running my applications along with offering some internal storage.

I understand. I like SAS, and currently use it. Hence the mention of Fujitsu's. They turned out to be the best fit for workstation use at the time. Not sure about now, but it wasn't that long ago. (Sep. '08).

SATA's great for massive capacity, but I need speed. My capacity requirements aren't that extreme, and can be accommodated with SAS.

WD's Velociraptors don't quite cut the quality I desire because they don't offer enough internal storage and they are more expensive than some SAS drives that are faster and offer more space.

As for size, meh. They compare to midrange (capacity) SAS (15k, not 7200 spindles). Pricing, depends. The backplane compliant versions are just too expensive. As an option, it might be worth running the numbers ($$$) on the cable connect models, and MaxUpgrade's sleds. Figure ~$250 per, including mounting. Slightly cheaper than the fastest SAS units. But the speed is no comparison to the Cheetah. Fujitsu's are also quicker IIRC, (need to check Toms or elsewhere on specific data for VR), but you pay for it. Just a lower cost idea. Not raw speed.

Let me clear something up: I am planning to RAID 5 the internal SAS drives alone. The external drobo setup will be completely separate. Each drobo also has only four drives in a special "beyond-RAID". This will ensure that the write hole issue doesn't exist in my setup. If I recall correctly, there is a RAID card for the internal drives that offers battery backup to avoid the cause of the issue.

Ah. That's definitely a nice way to go I think. Their "Beyond RAID" is very interesting. I wish ZFS/RAID-Z/RAID-Z2 was more widely available, but as my needs require the use of Windows as the primary OS, it's not yet possible.

Regarding the slowing down of daisy chaining, I don't think I would want to daisy chain any more than 2 of the devices. If the future Mac Pro does indeed have 2x firewire 800 ports, this would only allow for up to 4 drobos, which won't be enough for what I will end up storing. Now the question becomes if I can use a PCI express 2.0 card to add more firewire 800 ports. I plan on having a RAID card and two graphics cards (not doublewide), but Apple's website allows for 4x ATI cards in the current model, so I'm assuming there be at least this many in the future. With that note, it boils down to if the firewire cards intended for PCs will work with Macs, as I don't think there is a Mac only firewire PCI card.

You should be able to add a FW card. I presume you'd use a single graphics card, and a RAID card (presuming slot 2*). So 2 slots left, @ 4x on the current model.

I mentioned the use of internal ports on a RAID card though, as a higher speed alternative. Quite doable, and have assisted other members in setting them up. FAST. SATA based, so not as quick as a SAS rig. If the arrays are kept smaller, RAID 5's write hole issue won't be as bad. But not eliminated.
Playing ye olde statistics.

It was just something to consider. Going from say an ARC-1222, which would handle the 4 bays quite nicely, to an ARC-1680ix12,16, or even 24 would allow you to attach up to 28 drives (the number does not count the external 4 ports) without the need for SAS expanders. I use the ARC-1680ix12. Extremely happy with it. I wanted the ARC-1680ix16, but didn't truly need it. A bargain when you consider the features you get with the 1680 series IMO. You might want to note, that not all the 1680 gear have the same specs. Some use 1200MHz IOP's, while the more basic models use 800MHz IOP's. They also dump the 4 external drive port, and upgradeable cache. Very nice BTW. It can give some incredible reads.

You could attach up to 24 externally using the 1680ix24. Again, it's just an idea, as I have no idea of your budget.

Plenty of external enclosures that would be suitable, from pedestal to rack mounts.

If you decide to go this route, and wish to use SATA drives, please pay careful attention to Areca's HDD Compatibility List. Their SAS cards are picky with SATA drives. Not one consumer drive seems to work properly. Seagate and WD's enterprise drives do (save the RE3's, last I looked). At the time, we couldn't get them to work.

*Cable routing won't be fun. The best solution requires splicing the power conductors for additional length to reach basically anything beyond slot 4. Shorter cards make it even worse.

 

[geoffreak]

I'm planning to use the drobos to reduce the cost of storage offered by a rackmount solution. Using 4x WD RE3's I'm looking at less than a quarter the price of an equivalent sized rackmount solution. I might not have the speed offered by a rackmount, but I don't need it for storage. The internal drives are where the speed will be.

I've read that you can format a drobo in ZFS, but it only sees it as a single drive as all RAID is handled internally. For this reason, it really isn't necessary.

Just fyi, my budget is one of a computer enthusiast. I can't afford to spend $50k on computer parts, but I might be able to pull together $6k upfront for the computer as well as a couple drobos, with the intention being to expand to more as time goes on.

I was also wondering about Apple's stock SAS drives. Those seem to be reasonably priced for their size, but what really matters is performance.

PS: For those of you who are wondering, I am a serious multitasker and need all the speed and storage I can have in order to get more done. Currently on my MBP, I have open: 5 spreadsheets, 7 word documents, 20 firefox tabs, Dreamweaver, Fireworks, Illustrator, and iTunes. I have to constantly close programs and windows in order to free up memory and processing power. Whenever I want to do any video processing or detailed image work, I have to shut everything else down in order to process in a reasonable amount of time, thereby limiting my multitasking. I may be one of the few people that can easily max out a computer without running applications for the sake of applications.

 

[Trip.Tucker]

If I decided to end up going for a Mac Pro with pure speed, I would want to fill it with 4 SAS drives. I've heard that SAS drives are completely unrivaled in speed (15,000 rpm ), but does this really correlate to high speeds? Is Apple's RAID card the best solution for SAS drives? I would likely use RAID 5 setup for striping benefits as well as data redundancy.

Now, I'm a bit of a storage junkie, and SAS drives won't be a viable storage solution for everything I have already (and even more in the future), so external storage would be a must. I have a drobo already, and it is a much cheaper solution than getting something like an Xserve RAID, and firewire 800 is plenty fast for streaming video, even to multiple sources. My question is what will happen if I hook up 4, 5, or even 6 drobos? How does daisy-chaining many storage solutions affect speed? I won't be buying a Mac Pro until the next generation, and I'm assuming that there will be 2 firewire 800 ports, but will this even be much of an advantage? How about RAID striping across chained firewire?

Hope all these questions can be answered.

Maybe it's time to consider a miniSan in place of all those DROBO's you are considering. ....unless 15 drives is still not enough?

 

[geoffreak]

I don't know exactly what a miniSan is, but I'm assuming it's a type of rackmount storage? I wouldn't mind using a rackmount solution, but I would like to have a solution that is compatible with Apple's fibre channel card and I can purchase it under $2-3k without any drives. It would also need to be capable of adding drives on the fly as I will be adding them as I buy them. ZFS would also be a must just because of the fact RAID doesn't work well with the number of hard drives I'll be using.

 

[grue]

Better performance - No (if u install a Fast SSD)
Higher Storage - Yes
Cheaper - Definitely yes

Cheetahs in RAID will lay waste to SSDs in RAID when it comes to write speed.

 

[nanofrog]

I've read that you can format a drobo in ZFS, but it only sees it as a single drive as all RAID is handled internally. For this reason, it really isn't necessary.

It will do this whether an external box, or directly by a MP + Snow Leopard Server. Not going to be present in the client edition last I saw.

Just fyi, my budget is one of a computer enthusiast. I can't afford to spend $50k on computer parts, but I might be able to pull together $6k upfront for the computer as well as a couple drobos, with the intention being to expand to more as time goes on.

Maybe this will help.

SAN
DAS

What I was attempting to aim you towards, is DAS. All drives are attached to the RAID card. Additional cables and an external enclosure(s). Rather simple, and not extremely expensive for what it is, and the resulting speeds. No FC cards to add in, .... Most of the cost will end up in drives, and you add as you see fit.

For example, if you wish to go nuts, you could get a rack mount enclosure (24 bays), an ARC-1680ix24, and 6 external cables.

Enclosure = $1100 (hot swap, redundant PSU IIRC included)
Cables (1x SFF-8088 to SFF8088, 5x SFF-8088 to SFF-8087) = $450
RAID Card = $1100

So for $2650, you can go 24 external drives, and 4 internal, all using a single RAID card. No drives, but you add as many as you wish, up to the max.

You can also go in pieces. (Upgrade as you go strategy).
Let's say you want 12, maybe 16 drives max.
Get the following:
ARC-1680ix12 $760 (remember, it can actually run 16).
Cables (worse case; 1x SFF-8088 to SFF-8088, 2x SFF-8087 to SFF-8088) $225
Enclosures: You can go 1x 4bay + 1x 8 bay, or 3x 4bay (most expensive in the long run, but you can pay for 1 at a time). Lots of upgrade options.
Let's assume you need an 8bay box to start. Enhance E8MS. $550 (Note: Enhance E4/8 products are pedestal, not rack mount. They also match aesthetically with a MP).

This way would work to $1460 (no S/H included), + drive cost.

As mentioned earlier, if you use a SAS card, they are picky about SATA drives. Areca SAS cards will not run RE3's. They will however, run RE2's, and Seagate ...NS drives. I must really caution you on this, as returns are a PITA.

I don't know exactly what a miniSan is, but I'm assuming it's a type of rackmount storage? I wouldn't mind using a rackmount solution, but I would like to have a solution that is compatible with Apple's fibre channel card and I can purchase it under $2-3k without any drives. It would also need to be capable of adding drives on the fly as I will be adding them as I buy them. ZFS would also be a must just because of the fact RAID doesn't work well with the number of hard drives I'll be using.

Don't bother with Apple's offerings. Better stuff out there (3rd party). You might want to look at Atto Technologies as well.

The DAS may really be your best bet. I don't know enough specifics on how many drives you wish to start with, how many you want for expansion, or how you wish to divide interfaces (SAS + SATA). The mention of the RE3's has added a little more .

 

[giffut]

All ...

PS: For those of you who are wondering, I am a serious multitasker and need all the speed and storage I can have in order to get more done. Currently on my MBP, I have open: 5 spreadsheets, 7 word documents, 20 firefox tabs, Dreamweaver, Fireworks, Illustrator, and iTunes. I have to constantly close programs and windows in order to free up memory and processing power. Whenever I want to do any video processing or detailed image work, I have to shut everything else down in order to process in a reasonable amount of time, thereby limiting my multitasking. I may be one of the few people that can easily max out a computer without running applications for the sake of applications.

... this wonderful storage won´t help you with your slowing Macbook Pro. There you just can maximize memory depending on your machine and - for really speedying it up - replace the internal drive with a Intel X25 SSD drive. Here you could add a Sonnet eSat expresscard, grab atwo Sonnet Fusion 500P casings, which together will provide you with 10 drives maximum.

Same goes for a Mac pro, where you can delegate the VM cache to a fast SSD and use a VelociRaptor for boot/system/program drive. All the rest of it doesn´t give you really more speed, but just more storage.