How do I match read speed of Fusion drive with external storage?

[badlydrawnboy]

I'm about to buy a retina iMac and am trying to decide between 3 TB Fusion (which I have in current iMac) or 512/1 TB SSD + external TB (probably the OWC Thunderbay 4).

I've read that the read speed of the Fusion drive is very fast and difficult to match with an external drive set up, unless you use RAID-0 with 2-3 drives.

My question: what external configuration would I need to match the read speed of the Fusion 3 TB? I'm a little less concerned about write, because from what I've read read operations outnumber write by at least 10/1 for most activities.

 

[DotCom2]

I'm about to buy a retina iMac and am trying to decide between 3 TB Fusion (which I have in current iMac) or 512/1 TB SSD + external TB (probably the OWC Thunderbay 4).

I've read that the read speed of the Fusion drive is very fast and difficult to match with an external drive set up, unless you use RAID-0 with 2-3 drives.

My question: what external configuration would I need to match the read speed of the Fusion 3 TB? I'm a little less concerned about write, because from what I've read read operations outnumber write by at least 10/1 for most activities.

Excellent question.
Waiting to see what good answers come in.
I'm in the same dilemma.

 

[kendrickhphoto]

It really depends on how much you're willing to spend.

If you want an extremely fast drive that most internal SSDs including the PCI-E based ones in the iMac then go with the LaCie Little Big Disk

You could also go with the LaCie D2 with the SSD add on and get sort of an external Fusion drive. For 3 TB + SSD you're looking at $600 and for 6 TB + SSD you're looking at $800. It is Thunderbolt 2 and both of them over 1000 MB/s which is extremely fast for a good amount of storage.

The problem with both of those are you'd need a NAS or something similar to back it up to as they don't offer any kind of protection from drives failing.

 

[badlydrawnboy]

The LaCie D2 + SSD is an interesting option that I wasn't aware of, thanks.

I have an OWC Mercury Elite Pro with a 4 GB disk partitioned for Time Machine and bootable clone backup. It's USB 3.0. Could I hook that up to the LaCie D2?

How much noise does the LaCie make?

Another option I was considering was a RAID-0 with two or 3 4 TB drives, with the fourth drive used for backup. But I'm guessing this may not match the read speeds of the Fusion.

 

[Panch0]

LaCie D2 + SSD is not the same as Fusion. It will mount as two separate volumes. You have to divide whether a given file should be on the fast SSD volume or the slower HDD volume.

Fusion is basically the same thing - a small SSD + a large HDD. However, it displays as a single volume and Apple Pixie Dust decides where a given file should be for you . It will move frequently used files to the SSD portion so that the things you are most likely to request can be delivered at SSD like speed. The bulk of your data will still be on an HDD, and when those less frequently accessed files are requested, it is just a plain old single spindle HDD delivering them.

Going with an internal SSD + external storage instead of Fusion just means that you are choosing to manage your storage manually to some degree instead of letting the Fusion drive present your Fast and slow storage as a single volume that it tries to optimize for you.

What sort of files would you be putting on the external storage system if you decide not to go with the fusion drive? It is entirely possible that the data you would store there doesn't really need to be super fast - for instance, you can store HD Video on a single spindle HDD on USB2 and it will playback just fine.

 

[badlydrawnboy]

LaCie D2 + SSD is not the same as Fusion. It will mount as two separate volumes. You have to divide whether a given file should be on the fast SSD volume or the slower HDD volume.

Fusion is basically the same thing - a small SSD + a large HDD. However, it displays as a single volume and Apple Pixie Dust decides where a given file should be for you . It will move frequently used files to the SSD portion so that the things you are most likely to request can be delivered at SSD like speed. The bulk of your data will still be on an HDD, and when those less frequently accessed files are requested, it is just a plain old single spindle HDD delivering them.

Going with an internal SSD + external storage instead of Fusion just means that you are choosing to manage your storage manually to some degree instead of letting the Fusion drive present your Fast and slow storage as a single volume that it tries to optimize for you.

What sort of files would you be putting on the external storage system if you decide not to go with the fusion drive? It is entirely possible that the data you would store there doesn't really need to be super fast - for instance, you can store HD Video on a single spindle HDD on USB2 and it will playback just fine.

I was wondering about that, thanks for clarifying.

I would be storing Canon 5D3 RAW files on the external drive (along with music, videos, etc., but speed wouldn't matter there). That's where the fast read speed would be important.

I suppose I could just store the latest 6 months of files on the internal SSD, but I dislike the idea of having my images in separate places/catalogs. Makes managing them more labor intensive.

If I did go with a Thunderbay IV from OWC with RAID-0, how many drives would I need to put together in order to get 700 MB/s read speed? Or is that not the way to look at it?

 

[robgendreau]

I'm not sure that you need the speed; it is a matter of cost as well.

The reason the internal flash memory storage on an iMac or Mac Pro is so fast is that it is PCIe storage; that's an interface used for all sorts of stuff that requires a really really fast pipeline to the brains of the computer. All kinds of stuff use it. Here, it's flash memory. It's different than just a SSD slapped into a drive bay; those are SATA III interfaces, which are slower.

And all those internal SSD drives you see use SATA III. So if you put one in a Thunderbolt 2 enclosure, they still are limited by the SATA III speeds since that's how fast the drive can pump out data.

A RAID configuration can best this, because it's reading more than one disk at once to take advantage of that Thunderbolt 2 pipeline. The fastest I guess would be RAID 0 striping, so you'd have four drives of 1TB minimum for 1TB total storage, and at least one cold spare. And you'd need a backup of that, of course, since it's the least redundant.

But I'd consider whether you'd get much benefit from it if all you're doing is messing with a RAW file, even a very big one. If your software is behaving well and using RAM and/or scratch disk on the fast internal flash memory, it shouldn't have to be reading a lot from your external. Ask around, but it may be that the benefits (at least in speed; there are other benefits related to fault tolerance, etc) of a RAID vs just an external SSD are minimal for this type of work. Take at look at this analysis of RAID 0 SSDs vs single SSDs, and in particular at the application-oriented tests. Although the RAID was faster, and would be if you just copied those RAWs back and forth, picture imports were equally fast between the RAID and single disk.

http://www.tomshardware.com/reviews/ssd-raid-benchmark,3485-8.html

 

[badlydrawnboy]

I'm not sure that you need the speed; it is a matter of cost as well.

The reason the internal flash memory storage on an iMac or Mac Pro is so fast is that it is PCIe storage; that's an interface used for all sorts of stuff that requires a really really fast pipeline to the brains of the computer. All kinds of stuff use it. Here, it's flash memory. It's different than just a SSD slapped into a drive bay; those are SATA III interfaces, which are slower.

And all those internal SSD drives you see use SATA III. So if you put one in a Thunderbolt 2 enclosure, they still are limited by the SATA III speeds since that's how fast the drive can pump out data.

A RAID configuration can best this, because it's reading more than one disk at once to take advantage of that Thunderbolt 2 pipeline. The fastest I guess would be RAID 0 striping, so you'd have four drives of 1TB minimum for 1TB total storage, and at least one cold spare. And you'd need a backup of that, of course, since it's the least redundant.

But I'd consider whether you'd get much benefit from it if all you're doing is messing with a RAW file, even a very big one. If your software is behaving well and using RAM and/or scratch disk on the fast internal flash memory, it shouldn't have to be reading a lot from your external. Ask around, but it may be that the benefits (at least in speed; there are other benefits related to fault tolerance, etc) of a RAID vs just an external SSD are minimal for this type of work. Take at look at this analysis of RAID 0 SSDs vs single SSDs, and in particular at the application-oriented tests. Although the RAID was faster, and would be if you just copied those RAWs back and forth, picture imports were equally fast between the RAID and single disk.

http://www.tomshardware.com/reviews/ssd-raid-benchmark,3485-8.html

Thanks a lot for this. Maybe I'd be better off going with the 1 TB internal flash, and then getting a single 960 GB SSD in external enclosure. Then if I eventually need more storage space I could get another 960 GB SSD and put it in RAID 0. I already have a 4 GB HDD in a Mercury Elite Pro enclosure (USB 3.0) for backup.

What's the fastest single 960 GB SSD out there?

 

[hfg]

The read speed on a Fusion drive is only fast IF the data being read is actually on the SSD portion of the Fusion drive when accessed. If is comes from the hard disk portion, the read speed will be that of the hard disk (much slower). If the file is accessed a couple of times, the Fusion algorithm will attempt to put some/all of the file data blocks on the SSD for future accesses.

To duplicate this on an external drive, you would need to have an external SSD, which wouldn't be quite as fast as the internal PCI drive, but it would still be fast. For even faster speed, you could set up a RAID-0 SSD in an external Thunderbolt enclosure (i.e. Akitia Thunder Duo or LaCie LittleBigDisk) using a pair of standard SSD drives. If you need great capacity which is unaffordable with pure SSDs externally, you could create an external DIY Fusion drive in an external multi-drive JBOD enclosure using a small SSD and a hard disk of your choice. I would probably not use a 128GB SSD as Apple does, but go with a fast 256GB SSD so more data/files would be resident on the SSD portion when accessed.

For max speed ... get the latest LaCie Little Big Disk with the RAID-0 blade SSDs

 

[badlydrawnboy]

The read speed on a Fusion drive is only fast IF the data being read is actually on the SSD portion of the Fusion drive when accessed. If is comes from the hard disk portion, the read speed will be that of the hard disk (much slower). If the file is accessed a couple of times, the Fusion algorithm will attempt to put some/all of the file data blocks on the SSD for future accesses.

To duplicate this on an external drive, you would need to have an external SSD, which wouldn't be quite as fast as the internal PCI drive, but it would still be fast. For even faster speed, you could set up a RAID-0 SSD in an external Thunderbolt enclosure (i.e. Akitia Thunder Duo or LaCie LittleBigDisk) using a pair of standard SSD drives. If you need great capacity which is unaffordable with pure SSDs externally, you could create an external DIY Fusion drive in an external multi-drive JBOD enclosure using a small SSD and a hard disk of your choice. I would probably not use a 128GB SSD as Apple does, but go with a fast 256GB SSD so more data/files would be resident on the SSD portion when accessed.

For max speed ... get the latest LaCie Little Big Disk with the RAID-0 blade SSDs

Thanks. That's what I'm leaning towards now. Maybe 2x 500 GB Samsung 840 EVOs in a RAID 0 in a Thunderbay 4 enclosure, with a 4 GB HDD (that I already have) in there as well for backup. Then I can add a third 500 GB SSD eventually if I need more storage. It has taken me 3-4 years to get to 500 GB of photos, so I imagine this amount of storage capacity will be more than enough—and super fast.

 

[hfg]

Thanks. That's what I'm leaning towards now. Maybe 2x 500 GB Samsung 840 EVOs in a RAID 0 in a Thunderbay 4 enclosure, with a 4 GB HDD (that I already have) in there as well for backup. Then I can add a third 500 GB SSD eventually if I need more storage. It has taken me 3-4 years to get to 500 GB of photos, so I imagine this amount of storage capacity will be more than enough—and super fast.

That is what I have on my 2012 768GB SSD iMac ... I have a pair of 840Pro SSDs in RAID-0 with my Aperture Library on them, a 3 TB hard disk for archive stuff and movie files, and a 6TB drive for first-level Time Machine backups all in a Thunderbay IV. It works great and mimics the setup in my other system of a tower Mac Pro (I keep the synchronized).

For backup, I use a Synology NAS and a MacMini with Thunderbay IV RAID-5 as a TM NAS and Time Machine alternates between all 3 destinations.

 

[g4cube]

LaCie D2 + SSD is not the same as Fusion. It will mount as two separate volumes. You have to divide whether a given file should be on the fast SSD volume or the slower HDD volume.

Fusion is basically the same thing - a small SSD + a large HDD. However, it displays as a single volume and Apple Pixie Dust decides where a given file should be for you...

In this article over at http://www.barefeats.com/hard191.html, Rob-ART provides some info where he configured the d2+SSD upgrade as a Fusion drive. Pretty incredible performance.

Granted, LaCie offers this as one external chassis with 2 separate volumes inside - SSD + HDD. Rob's incantations turn it into a Fusion drive with the caveat that this is not supported by Apple or LaCie. This is however equivalent to what others have done for "roll-your-own" Fusion configurations.

I've done something similar with a refurb Little Big Disk v1, where I replaced one of the 2 HDD with a Samsung EVO SATA SSD. Inside are a 2TB HDD and a 256GB SSD. Of course performance is great for the first 256GB of storage.

 

[robgendreau]

It has taken me 3-4 years to get to 500 GB of photos, so I imagine this amount of storage capacity will be more than enough—and super fast.

Well with that 5K monitor you'll want photos and videos to match, which means new cameras with bigger formats, which means bigger RAW files, which means you'll double your storage needs really fast

Just a question of whether you can fill your storage as fast as your credit card balance

The benefit of external is you could start slower, and see with some skillful management if you really need the pricier options. Rarely used files sitting on SSDs don't make much sense.

 

[badlydrawnboy]

Well with that 5K monitor you'll want photos and videos to match, which means new cameras with bigger formats, which means bigger RAW files, which means you'll double your storage needs really fast

Just a question of whether you can fill your storage as fast as your credit card balance

The benefit of external is you could start slower, and see with some skillful management if you really need the pricier options. Rarely used files sitting on SSDs don't make much sense.

I've got a Canon 5D3 full-frame, not planning to go higher res than that (though I know the megapixel war will continue). I really like the convenience of having my entire photo library in one catalog and on one drive, so a RAID-0 with 2x500 GB would be a nice solution, even if it is probably overkill.

 

[robgendreau]

I've got a Canon 5D3 full-frame, not planning to go higher res than that (though I know the megapixel war will continue). I really like the convenience of having my entire photo library in one catalog and on one drive, so a RAID-0 with 2x500 GB would be a nice solution, even if it is probably overkill.

That would certainly work if you can fit all your stuff onto 500GB, plus allowing for some overhead.

But you can have one library or catalog (dunno what software you're using) and the photos on many different disks if you're using any DAM that supports referenced images (I think all do). So you get the benefit of speedy previews, searches, etc etc with the more spacious HDDs for photos you aren't currently using. Especially if you catalog is on the internal PCIe based flash, since it's probably the fastest.

 

[badlydrawnboy]

That would certainly work if you can fit all your stuff onto 500GB, plus allowing for some overhead.

But you can have one library or catalog (dunno what software you're using) and the photos on many different disks if you're using any DAM that supports referenced images (I think all do). So you get the benefit of speedy previews, searches, etc etc with the more spacious HDDs for photos you aren't currently using. Especially if you catalog is on the internal PCIe based flash, since it's probably the fastest.

Ah, thank you for that tip. I wasn't aware of this. I use Lightroom. Right now my catalog and images are stored on my 3 TB Fusion drive. But if I were to get a retina iMac with 512 GB internal flash, I should store the catalog on that with the photos on the external drive? And I could theoretically have the most recent photos in the catalog stored on a RAID-0 with SSDs, and the older files on an HDD?

Though what I'm thinking of doing now is just going with the smaller internal drive (512 GB) since I will only store apps and OS X on it, and then getting a pair of Samsung 840 1 TBs and putting them in a RAID-0 in a Thunderbay IV, along with a couple of 4 GB HDDs for backup and Time Machine. I think I could have a third HDD for backup that I can swap out for one of them periodically for redundancy.

 

[joema2]

Maybe I'd be better off going with the 1 TB internal flash, and then getting a single 960 GB SSD in external enclosure...

The cost of 1TB internal SSD plus external 960GB SSD plus OWC enclosure is very roughly equal to the cost of a 256GB SSD iMac plus a Thunderbolt 2 8TB Pegasus R4. In RAID 5 the formatted capacity is about 6TB. The FD is a little faster on *cached* reads, but much slower on writes. The R4 is more consistently fast over a wide range of tasks.

 

[badlydrawnboy]

The cost of 1TB internal SSD plus external 960GB SSD plus OWC enclosure is very roughly equal to the cost of a 256GB SSD iMac plus a Thunderbolt 2 8TB Pegasus R4. In RAID 5 the formatted capacity is about 6TB. The FD is a little faster on *cached* reads, but much slower on writes. The R4 is more consistently fast over a wide range of tasks.

Thanks. I'm now thinking of a 512 GB internal flash + ThunderBay IV w/ 2x 1TB SSDs in a RAID-0. With Thunderbolt 2, it's my understanding that this external SSD RAID will be faster than the internal flash storage. Is that correct?

 

[joema2]

....With Thunderbolt 2, it's my understanding that this external SSD RAID will be faster than the internal flash storage. Is that correct?

Sorry, I don't know. That does sound like a very fast configuration.

However -- I have a 5D3 and shoot lots of video and raw stills. I would not over-emphasize the I/O aspect, because many common tasks are CPU bound. E.g, in Lightroom if you import CR2 files and build 1:1 previews, it is more CPU-bound than I/O bound.

You can examine the CPU vs I/O state of these activities using the utility iStat Menus.

You want fast I/O but only to the extent it's not the weakest link. Once over a certain threshold, further I/O performance increases don't make much difference -- you are waiting on CPU, GPU, etc.

Whether you are running RAID5 or SSD or whatever, it also should be backed up.

 

[Bryan Bowler]

OP,

This thread has been very helpful to me since I'm trying to sort through my options as I get ready to buy a new 5K iMac and a Thunderbolt 4 RAID system. I do a ton of photo and video work.

Anyway, I did want to point out one thing. I know your super-smart and already know this, but perhaps this may help others that read this thread.

Thunderbay IV is the OWC's Gen 1 array that uses the Thunderbolt 1 standard.
Thunderbay 4 is the OWC Gen 2 array that uses the Thunderbolt 2 standard.

It's easy to confuse both of them when typing, and I've seen that throughout this thread, but there is a very important difference between the two in terms of speed. Just wanted to point that out so an unknowledgeable user doesn't accidentally purchase the Thunderbay IV if they want the TB2 standard!

Bryan

 

[badlydrawnboy]

OP,

This thread has been very helpful to me since I'm trying to sort through my options as I get ready to buy a new 5K iMac and a Thunderbolt 4 RAID system. I do a ton of photo and video work.

Anyway, I did want to point out one thing. I know your super-smart and already know this, but perhaps this may help others that read this thread.

Thunderbay IV is the OWC's Gen 1 array that uses the Thunderbolt 1 standard.
Thunderbay 4 is the OWC Gen 2 array that uses the Thunderbolt 2 standard.

It's easy to confuse both of them when typing, and I've seen that throughout this thread, but there is a very important difference between the two in terms of speed. Just wanted to point that out so an unknowledgeable user doesn't accidentally purchase the Thunderbay IV if they want the TB2 standard!

Bryan

Thanks for clarifying. It is an important difference; no idea why OWC didn't just call it the Thunderbay 5 or something else to make the difference more obvious.