SSD + RAID 0 + Home Brew Fusion Drive (for fun and profit...)

Discussion in 'Mac Pro' started by panjandrum, Jul 31, 2017.

  1. panjandrum macrumors regular

    panjandrum

    Joined:
    Sep 22, 2009
    Location:
    United States
    #1
    Just received a massive donation of Mac Pros (4,1s, quickly becoming 5,1s) which allowed me to do an amusing test:

    I created a RAID 0 array of 2x1TB WD Black drives, added a small 120GB SSD, and then created a Fusion drive out of the SSD + RAID array. (No worries about data-loss, it's just a test bed and it gets a nightly bootable CCC clone to single 2TB internal drive anyway).

    I googled about a bit to see if anyone else had tried this, but couldn't really find anything.

    I'll try to remember to stop back in a few months with a reliability report.

    Next test? To see if slapping 2x2TB SSHDs into a Raid 0 and then making those part of an SSD+RAID 0 fusion drive will result in a massive Gnab Gib of data corruption... or not...

    (Yes, I know it is silly and mostly pointless, I just want to see if it will work. So far, yes, it boots and functions as it should, but I have no idea if it will stay that way.)
     
  2. Blakehoo macrumors member

    Blakehoo

    Joined:
    Aug 4, 2016
    #2
    How'd you do it...?
     
  3. owbp macrumors 6502a

    owbp

    Joined:
    Jan 28, 2016
    Location:
    Belgrade, Serbia
    #3
    @panjandrum , I assume you've made Recovery HD on SSD before merging it to Fusion with RAID array?
    Can you boot into it (for the purpose of disabling SIP)?

    I'm using 2x1TB RAID 0 for User "folder" and 120GB SSD for macOS, so I might try just that with my setup but I have to disable SIP for audio over HDMI.
     
  4. panjandrum thread starter macrumors regular

    panjandrum

    Joined:
    Sep 22, 2009
    Location:
    United States
    #4
    I wouldn't suggest trying it unless it's a machine that's either backed-up routinely or doesn't have anything important on it. My guess it that Apple never thought anyone would be silly enough to try it and that things are going to go to heck soon... And, since I keep a backup drive in the system anyway, I have the recovery partition on that.

    Blakehoo: Just in case you aren't aware of this, doing anything like creating RAIDs or Fusion drives will completely wipe all data from the drives in question... But assuming you know that, it is easy, just snag the instructions (I use the set by Other World Computing, they are pretty concise) but before you start the process to create the Fusion "drive" create a RAID 0 out of 2 or more other drives. When you get to this step:

    diskutil coreStorage create FUSION /dev/disk1 /dev/disk2

    enter the disk# of your RAID as the second "disk" (where it says "disk2" in the line above, enter the disk# assigned to your RAID, not the disk# assigned to any single disk in the system).

    I used it through the day today and so far so good, with extremely good performance.
     
  5. Draeconis macrumors 6502a

    Joined:
    May 6, 2008
    #5
    A guy on here asked me about creating a Fusion Drive RAID 1 setup, which I eventually got working.

    It was bootable, and the drives were literally mirrored (SSD writing happened on the other SSD, etc).

    I was skeptical that AppleRAID would accept logical volumes, but it did!
     
  6. eksu macrumors member

    Joined:
    Aug 3, 2017
    #6

    Can you share any more information on how you accomplished this? Very interested. Thanks.
     
  7. h9826790 macrumors 604

    h9826790

    Joined:
    Apr 3, 2014
    Location:
    Hong Kong
    #7
    I bet it's just make two Fusion Drive, and then RAID them together in disk utilise.

    Apple software RAID almost accept any writable volume. I tried RAID a USB stick to a SSD, a drive to a partition, a SSHD to a HDD. It accept whatever combination I throw into it. Of course, the combination may degrade the performance or waste lots of space, or totally meaningless. But in my own test. Any combination work. I never use Fusion Drive. But I assume disk utility will treat it just like a single drive and accept it as a part of the RAID.
     
  8. eksu macrumors member

    Joined:
    Aug 3, 2017
    #8
    Ideally I would like it to behave as if you raid the SSD's together, raid the HDD's together, then make a logical volume with those. I wonder if this gets around putting the recovery partition on raid.

    Functionally, you'd want it to act like RAID 10, not RAID 01.
     
  9. goMac macrumors 603

    Joined:
    Apr 15, 2004
    #9
    I had this configuration for a few years. One of the macOS updaters (I think El Capitan) flipped out when it saw that configuration. It might have been fixed since then, but I ended up moving back to a single HDD as part of the CoreStorage volume, and then eventually to a pure SSD setup.
     
  10. h9826790 macrumors 604

    h9826790

    Joined:
    Apr 3, 2014
    Location:
    Hong Kong
    #10
    My understanding is as follow.

    Assuming you use the same SSD and HDD to create fusion drives. Due to the fusion drives are managed by the same OS, using the same logic, even RAID 1 after fusion, the data should still like SSD mirroring to SSD, HDD mirroring to HDD. Of course, the reason behind is very different.

    But once one of the drive fail, one of the whole fusion drive will break, and when you replace the new drive, you must re-create the fusion, and let RAID 1 to handle the data sync. At that point, the data is no longer SSD mirroring to SSD, and HDD mirroring to HDD. But can only let OS to decide how to store data on the new fusion drive.
     
  11. Draeconis, Aug 3, 2017
    Last edited: Aug 4, 2017

    Draeconis macrumors 6502a

    Joined:
    May 6, 2008
    #11
    So I created a test system with two SSDs and two HDDs. From this, you create two CoreStorage volumes from an SSD/HDD pair, which CoreStorage will identify as being able to utilise Fusion and work accordingly.

    So, you'd have (roughly);
    Physical Drives
    disk0
    disk1
    disk2
    disk3

    Logical Drives (Fusion)
    disk4
    disk5

    You'd then tell appleraid to mirror disk4 and disk5. Even though they're logical volumes spanning multiple disks, appleraid accepts this, and creates disk6, which is where your machine boots from.

    This setup can suffer one disk failure, and boot perfectly fine, but beyond that the setup will break. If a disk drops out, its best to drop out of the mirror, rebuild the fusion disk from a spare, and re-mirror.

    It has to be done this way; CoreStorage cannot create Fusion Drives from an appleraid volume, and you probably wouldn't want that anyway, since any single drive failure would take your system out completely.

    It's a nice test, and certainly interesting to try out. In practice, I would assume macOS installers would throw a fit trying to do anything with it (especially with APFS in the near future). If it's something you want, then similar when a drive fails, if you wanted to upgrade macOS version; backup your system, break the mirror, upgrade, and re-create. Also, HFS+ is garbage, so creating something like this, while interesting, I wouldn't trust. Also, macOS doesn't warn you when a disk drops out, so if you're cognisant enough to create this in the first place, then you'd be wise to write something to monitor the health of your system.

    Obviously, this is only simple when your SSD and HDD volumes match. The chap asking me about it had SSDs that were a different size. I recommended he partition the bigger SSD to match the smaller one, so as to simplify it. CoreStorage doesn't care what sizes the underlying disks are; it creates a logical volume spanning both volumes of the total size. You would mirror this to another CoreStorage logical volume of a different size, where the mirror will take the smaller size. However, if the underlying disks don't match, and mirroring seems to duplicate the write pattern, when Fusion Drive A writes something to the SSD that Fusion Drive B can't because it's smaller, then I don't know what happens.

    What I haven't tried is appleraid mirroring two APFS logical volume fusion drives.. that would be more interesting, since APFS is (hopefully) far more resilient to failure than HFS+ is.
     
  12. h9826790 macrumors 604

    h9826790

    Joined:
    Apr 3, 2014
    Location:
    Hong Kong
    #12
    The whole setup should still allow 2 drives failure, however, must be within the same fusion drive. E.g.

    Fusion drive A consist of SSD A + HDD A

    Fusion drive B consist of SSD B + HDD B

    Then RAID them together (RAID 1)

    If SSD A fail, the FD A failed, but RAID 1 still works.

    Of SSD + HDD A failed, FD 1 fail, RAID 1 should still work. Of course, after SSD A failed, HDD A should do nothing at all. Even it's still working, but same as failed in the data point of view.

    But if SSD A + HDD B fail, then whole Fusion Drive failed and unable to recover data.

    Therefore, if possible, form 2 SSD in a RAID 1. And another 2 HDD in RAID 1. And then fusion them together will have better redundancy. This allow the fusion drive continue to work in case SSD A + HDD B failed. However, AFAIK, we can't create fusion drive in this way.

    For difference size SSD, I also believe that make a partition on the larger one to match the smaller one is the best way to go.

    I am sure the RAID 1 can accept 2 fusion drive that formed by 4 different size drives. In this case, the performance should be ultimately limited by the HDD performance that on the fusion drive which has less SSD capacity. Just like RAID 1 a SSD with a HDD. We simply waste the SSD's performance, but the RAID 1 should work as expected.
     
  13. Draeconis macrumors 6502a

    Joined:
    May 6, 2008
    #13
    So, at a risk of repeating myself, you cannot create a CoreStorage logical volume from an AppleRAID volume. So what you propose is impossible.
     
  14. panjandrum thread starter macrumors regular

    panjandrum

    Joined:
    Sep 22, 2009
    Location:
    United States
    #14
    I did it the opposite way I think most people assumed I did: I built a single RAID 0 out of 2 drives FIRST, and then I created the FUSION (wait, do we need that to be all caps?) drive using a single small SSD and the logical volume created by the RAID setup. In theory this produces a FUSION which reads the SSD (not in RAID) plus the logical RAID 0 volume.

    This worked without any errors during the FUSION drive setup procedure. However, we don't know about real-world reliability at this point. It definitely seems to be very quick; to see if I'm getting real RAID 0 speed once I hit the spinning drives I will go ahead and create a second internal RAID 0 out of 2 more drives and use those for CCC instead of the single drive I currently use. Since I have far more data on the FUSION+RAID drive than the SSD alone can handle, if I monitor drive throughput I should be able to see if I'm getting 2-drive RAID 0 performance (benchmarking software probably wouldn't do a good job due the the FUSION setup, but cloning 1.2TB will certainly give me a good idea what's going on). I won't have time to do this right away, but I'll try to remember to get to it in the next few weeks.

    So here is an example setup of the way I did it:

    Let's pretend I have only 3 drives, the 1x120GB SSD and 2x1TB HDD, and that when I run diskutil list they are listed as disk1, disk2 & disk3.

    I create the RAID 0 out of the entire disks2 & disk3, which produces a logical volume RAID 0 array, let's say it is disk4.

    The command to then create the FUSION drive would be:

    diskutil coreStorage create FUSION /dev/disk1 /dev/disk4

    I really didn't expect it to work, but it did, and is still working just fine. Zero problems since my OP and good performance.
     
  15. panjandrum thread starter macrumors regular

    panjandrum

    Joined:
    Sep 22, 2009
    Location:
    United States
    #15
    I finally had time to stick 2 more drives in there and make them a second RAID 0 array. When copying sufficient data that I'm sure I'm not just hitting the SSD portion of my SSD+RAID 0 FUSION setup (for example, when using CCC to clone everything), I'm definitely getting RAID 0 speeds off the rotational drives. Still no problems at this point, which is a pleasant surprise considering I didn't even expect this to work, at all, in the first place. Disk Utility's First Aid feature consistently reports zero problems. So, I guess, if you use a FUSION system and want to maintain a bit more speed when you are moving data around on HDDs rather than the SSD portion, this might be a way to go. I still wouldn't recommend it unless you are as backup-paranoid as I am (which is extremely...)
     
  16. Blakehoo macrumors member

    Blakehoo

    Joined:
    Aug 4, 2016

Share This Page