Mac Pro - Bootable RAID Cards

[drndown2008]

Sorry I didn't post enough info:
•Boot requirements (some cards can boot OS X, but once the firmware is set to EFI, it can no longer boot any other OS).

Hope to boot straight into VMWare ESXi. Can have another drive with OSX on it if needed. Booting into ESXi via option button now.

•OS requirments (driver support)

Just ESXi, unless OS X is required (maybe just when a RAID configuration change is needed??). I was hoping (naively?) that with hardware RAID, drivers wouldn't be needed.

•Software/system usage

Will run some VM's in VMWare ESXi. Lightly used -- sort of a lab environment.

•Throughput requirements

Hmmm, don't know. Not all that high compared to what many of you are doing

•Capacity requirements

Plan on mirroring two 640GB drives to old ESXi itself, and the VMs. Will use another drive to run OS X or Windows if needed to configure the RAID array.

•Redundancy requirements

Just two disks. Someday maybe four, but that day is very far off, and doubtful.

Thanks

 

[nanofrog]

Hope to boot straight into VMWare ESXi. Can have another drive with OSX on it if needed. Booting into ESXi via option button now.

Hmm... I'm lookes like you'd need a hardware RAID controller, and leave the firmware as BIOS (software implementation would only work under OS X, unless that's the only OS you need the RAID for).

Just ESXi, unless OS X is required (maybe just when a RAID configuration change is needed??). I was hoping (naively?) that with hardware RAID, drivers wouldn't be needed.

I couldn't find enough about ESXi and RAID cards, so I'd err on the side of caution, and presume each OS would have to access the card and created array/s directly (means driver support for each intended OS to be used).

Will run some VM's in VMWare ESXi. Lightly used -- sort of a lab environment.

I've still no idea of throughput requirements, though assuming software development, it's not that much. Say ~400MB/s is more than sufficient IMO (reads, writes are always slower).

Hmmm, don't know. Not all that high compared to what many of you are doing

See above. From what you're posting, it's not that high.

Plan on mirroring two 640GB drives to old ESXi itself, and the VMs. Will use another drive to run OS X or Windows if needed to configure the RAID array.

It sounds like you need to contact VM Ware and see if ESXi can access RAID cards directly (I couldn't find it).

The answer is critical IMO, so do yourself a favor, and make the call (I'd think ESX can, as it's using Linux as it's base OS, but ESXi I'm not sure can do this).

Just two disks. Someday maybe four, but that day is very far off, and doubtful.

You just want a 2 disk array (RAID 1 from what I can tell), or do you want the array to be able to handle the loss of 2x drives and remain functional?

 

[mac-mac-mac]

This has been a great thread to follow. Please don't flame right away, since this is my first post.

I have a MP 08 with Areca 1680-ix and connect to external RAID5 thru a Areca-8020 expander over miniSAS cable 8088.

I would like to move this setup as my backup setup to a "hackintosh"-based computer. This is similar to what nanofrog mentions in post number 6 of this thread. I have setup a few hackintoshes for fun, but never could properly setup the Areca-1680-ix card in them.

Nanofrog:
Given the physical installation (space & cables), I opted to build. Currently under way. Just waiting for Intel to release the Xeon 5500 series/3500 series parts. And mother boards of course.

I'll just have to deal with hacking it to make OS X work.

Nano, have you moved forward on CPU/mobo setup for your storage?
Can your computer recognize Areca (especially the 1680) card? Please share your experience.

 

[nanofrog]

I have a MP 08 with Areca 1680-ix and connect to external RAID5 thru a Areca-8020 expander over miniSAS cable 8088.

Just out of curiosity, why the SAS expander?

Nano, have you moved forward on CPU/mobo setup for your storage?
Can your computer recognize Areca (especially the 1680) card? Please share your experience.

Yes and Yes.

It's easy to get it running in a PC, and not much more difficult in a MP (difference is in cabling, drive mounts, and firmware if you need to boot). Otherwise, it's drivers.

Did you change the firmware back to BIOS on the Areca?

If not, you'd need to do so (you can update it in the MP, but not with Safari last I tried - best to use Firefox), load the drivers during the OS install (assuming clean, if not, it's still quite doable, and the proceedure is in the manual), and you'll have access to it.

If you need to boot off of the array (which it doesn't seem like you'd need, given it's being pushed to backup purposes), you'd have to get into the system firmware and select it as the boot location. Again, rather easy to do.

Beyond this, I'd need further details.

 

[drndown2008]

I couldn't find enough about ESXi and RAID cards, so I'd err on the side of caution, and presume each OS would have to access the card and created array/s directly (means driver support for each intended OS to be used).

My assumption is that a hardware RAID controller would be invisible to the OS, so as long as I can get the array setup, drivers won't be an issue. And I'm willing to have a separate OS X and/or Windows drive to use just for configuring the array. Does that sound reasonable?

It sounds like you need to contact VM Ware and see if ESXi can access RAID cards directly (I couldn't find it).

I did find a hardware compatibility guide, but they assume PC hardware, BIOS, etc, so I suspect all bets are off with trying to get this working in the Mac Pro (it does work great -- without RAID so far)

You just want a 2 disk array (RAID 1 from what I can tell), or do you want the array to be able to handle the loss of 2x drives and remain functional?

RAID 1 -- if a drive fails, I'll immediately replace it and rebuild the array.

What I _really_ wish is that I could just get into the RAID card via Ctrl-H at boot up, configure the array in BIOS, and let it rip. But that's a no-go on the EFI-based Mac as far as I understand.

And thank you nanofrog for your time -- I really appreciate it.

 

[nanofrog]

My assumption is that a hardware RAID controller would be invisible to the OS, so as long as I can get the array setup, drivers won't be an issue. And I'm willing to have a separate OS X and/or Windows drive to use just for configuring the array. Does that sound reasonable?

Well, to do what you want, I'm not sure it will work in an MP due to the fact it uses EFI based firmware, and you can't access it.

I'm presuming you want the RAID 1 for the EXSi, which would load first, then any OS's. To do this, the boot location would normally be set in the system's firmware, which you can't do on an MP. You also have to get firmware access to the card to create the array (actually done before attempting to install EXSi), which you won't be able to do IIRC (but you would be able to create the array under another OS via the Web Browser).

Setting the array as the boot location for EXSi is the hard part, and without BIOS, I'm not sure it's possible. You could try to go through the support links for that product,but I'm not sure if someone else has found a way. Worth a shot though.

I did find a hardware compatibility guide, but they assume PC hardware, BIOS, etc, so I suspect all bets are off with trying to get this working in the Mac Pro (it does work great -- without RAID so far)

This is my opinion as well. It's the lack of access to the EFI firmware that's the problem.

 

[mac-mac-mac]

This relates to posts 53 and 54 in this thread.

Nano, I use the SAS expander Areca-8020 in order to have hard drives in one exclosure in the adjacent closet space. The connecting cable is a 4 meter 8088-8088 miniSAS cable. There are two reasons for this setup. One, I wanted only one cable to 15 drives in 2 separate RAID arrays, so I had to go externally. Second reason was that the noise level at my MP is extremely low since the hard drives and fans are far away.

My most burning question is that I tried to repeat this setup on a hackintosh. The latest build was Gigabyte EP45UD3P based (this link). I couldn't make this hackintosh recognize the Areca-1680-12-ix card? Do I understand from your post that this card needs to flashed with BIOS/PC firmware, as opposed to current Mac firmware. I did similar test with Areca 1300-16ix card and this card also was not recognized. I believe that if the OSX86 can recognize the card the rest should work out great. Could it be that the OSX86 hardware "disguises" block the areca card from being recognized? Are there any apple.com.plist or .kext tricks for areca cards?

Thank you for any suggestions.

 

[nanofrog]

Nano, I use the SAS expander Areca-8020 in order to have hard drives in one exclosure in the adjacent closet space. The connecting cable is a 4 meter 8088-8088 miniSAS cable. There are two reasons for this setup. One, I wanted only one cable to 15 drives in 2 separate RAID arrays, so I had to go externally. Second reason was that the noise level at my MP is extremely low since the hard drives and fans are far away.

It gets you past the issue of running cables out of a PCI bracket (SFF-8087 to SFF-8088 cables that take internal ports directly to the enclosure = sweet for what they do), and also allows for the additional length (1.0m limit otherwise for SATA drives makes enclosure placement much more difficult, and noisy).

My most burning question is that I tried to repeat this setup on a hackintosh. The latest build was Gigabyte EP45UD3P based (this link). I couldn't make this hackintosh recognize the Areca-1680-12-ix card?

Yes.

Do I understand from your post that this card needs to flashed with BIOS/PC firmware, as opposed to current Mac firmware. I did similar test with Areca 1300-16ix card and this card also was not recognized. I believe that if the OSX86 can recognize the card the rest should work out great. Could it be that the OSX86 hardware "disguises" block the areca card from being recognized? Are there any apple.com.plist or .kext tricks for areca cards?

Yes, you need to flash the firmware back to BIOS. Then get into the system firmware (Gigabyte board), and set the boot location to the array (really obvious vs. a single drive ).

It can be done. In the BIOS images , take a look at screens 5 & 6 IIRC (6 in particular, where it says "4. Bootable Add-in cards", as that's where I'd think you need to be. I've not used that board before, but I know it's possible.

There aren't any changes that causes the card to be "hidden", unless the firmware is the wrong specification for the board used (i.e. card flashed with EFI when it needs to be BIOS).

Also, Areca provides drivers for OS X, so use those. They will work.

 

[mac-mac-mac]

Thank you nanofrog. I am eager to implement your points. I will be away from my machines for a few days. Can't wait...

 

[nanofrog]

Thank you nanofrog. I am eager to implement your points. I will be away from my machines for a few days. Can't wait...

Good luck with it, and let me know how you get on with getting the board to recognize the card.

BTW, what part of Florida are you located (I'm from Orlando originally)?

 

[drndown2008]

I'm presuming you want the RAID 1 for the EXSi, which would load first, then any OS's.

There will just be a single set of mirrored disks which will hold ESXi and the VM images. So as far as the MP is concerned, it would only see one disk (I think). So if I can just find a bootable RAID controller, I thought it would 'just work'.

You also have to get firmware access to the card to create the array (actually done before attempting to install EXSi), which you won't be able to do IIRC (but you would be able to create the array under another OS via the Web Browser).

The plan was to have a second drive with OS X/Windows on it. I'll boot to that disk, create the array, and then shutdown. Pull the OS X/Windows drive out, and then (hopefully) boot to the array since it would be the only available drives.

Basically I have a wonderful piece of hardware and want to use ESXi with it. I hate the idea of selling it and getting a PC, but I might have to

 

[nanofrog]

There will just be a single set of mirrored disks which will hold ESXi and the VM images.

This much is clear.

So as far as the MP is concerned, it would only see one disk (I think).

Generally speaking, if you get the array established, and the OS see's it, then it appears as a single logical disk.

The issue is, the EFI based system, when the ESXi is expected to run on BIOS based systems. This isn't a problem with single drives (what you're doing now), but the RAID aspect is changing things with the specific version you're using.

My instincts tell me that you'd need to get the ESX version (Linux based, but you have to pay for it) to do what you're trying to do.

So if I can just find a bootable RAID controller, I thought it would 'just work'.

No, it won't, as you can't specify in the MP's firmware that the card is meant to be a bootable location.

If the system were BIOS base, then Yes, it should be possible (assuming there's no issues with that flavor from VMWare).

Beyond that (specific version you're trying to use), it's an experiment = No Guarantee's of what will/won't happen. Up to you if you want to be the guinea pig.

The plan was to have a second drive with OS X/Windows on it. I'll boot to that disk, create the array, and then shutdown. Pull the OS X/Windows drive out, and then (hopefully) boot to the array since it would be the only available drives.

It won't work, unless the system sees the array as the bootable location though. That's what I'm trying to get across to you. Unfortunately, the EFI based firmware used in Macs is a problem most likely (I'd be amazed if it works, since you can't get into the EFI and set the boot location = array as you can with other boards).

Basically I have a wonderful piece of hardware and want to use ESXi with it. I hate the idea of selling it and getting a PC, but I might have to

You may have to, but I think there's another possibility = buy the Linux OS equiped version (ESX version). It's worth contacting VM Ware IMO to see what they say. You may have to request/kindly insist to speak with an engineer, but someone should be able to tell you the specifics on that version.

 

[drndown2008]

No, it won't, as you can't specify in the MP's firmware that the card is meant to be a bootable location.

Maybe that's what I don't understand. Right now there is a single disk drive and MP boots from it (well, sort of...). With the RAID adapter in place, doesn't it present the hardware with a single hard disk as well? Especially if it's a 'mac-bootable' (EFI-based) RAID adapter?

 

[mac-mac-mac]

Sorry others, I cannot send PMs yet, this for nanofrog - Orlando, FL here, too!

 

[nanofrog]

Maybe that's what I don't understand. Right now there is a single disk drive and MP boots from it (well, sort of...). With the RAID adapter in place, doesn't it present the hardware with a single hard disk as well? Especially if it's a 'mac-bootable' (EFI-based) RAID adapter?

The system still has to point to it as a boot location, or it's ignored. That's why you need access to the firmware, so you can set the array as the boot location. But with the MP, there's no way to do this, so the system doesn't know to go there, and load anything.

It can however, use drives attached to the SATA ports directly on the logic board.

Sorry others, I cannot send PMs yet, this for nanofrog - Orlando, FL here, too!

I miss it for the most part, but not the traffic.

 

[drndown2008]

The system still has to point to it as a boot location, or it's ignored. That's why you need access to the firmware, so you can set the array as the boot location. But with the MP, there's no way to do this, so the system doesn't know to go there, and load anything.

I'm sorry for being dense, but does that mean it's impossible to boot from a RAID drive on MP? I thought I saw some RAID controllers that advertised being 'Mac-bootable'. But if it really is true that an MP simply cannot boot from a RAID array, then I'm at the end of my research

 

[nanofrog]

I'm sorry for being dense, but does that mean it's impossible to boot from a RAID drive on MP? I thought I saw some RAID controllers that advertised being 'Mac-bootable'. But if it really is true that an MP simply cannot boot from a RAID array, then I'm at the end of my research

Some can. To do it, they have to be flashed with the EFI boot code available. Areca and Atto offer such cards. Drivers are then loaded during the boot process (they're still needed for the OS to access the card once the Kernel is loaded).

But in the case of the VM software you're using, I'm not sure if it will work in this instance, as it's loaded prior to an OS and you're unable to set the boot location in the system's firmware (no access; directly or via utility, as Apple's never provided it in any of the Intel based systems).

That's the snag.

I can guarantee a software array won't work (OS dependent, and only good for the OS it's created under). If the system will allow the array to become available as an option in the boot menu, it might.

You can guinea pig this (at your time and expense), but VM Ware's not validated their software on a Mac Pro, let alone a RAID equiped system (software or hardware).

If it were a PC, you can access the system firmware, and set the array as the boot location. It might seem like a small difference, but it's major, and really does mean the difference between failure and functional.

That's why I think the ESX (Linux equiped variant) may be the required version to do what you want (of course you'll have to pay for it). You need to call VM Ware and see what they have to say on the matter, as they wrote the software (see what they say on both versions).

You can call Apple as well, as it's worth a shot. But somehow, I doubt they'd be of much help (you'd need to speak with an engineer higher up the chain - way up really, as the usual support staff will be stumped by this - it's really specific in the hardware/firmware interaction).

 

[drndown2008]

Drivers are then loaded during the boot process (they're still needed for the OS to access the card once the Kernel is loaded).

Ahhh, so maybe that's what I'm not understanding. Does that mean if I have a RAID array built with a 'bootable' RAID controller, the hardware still needs to at least load the kernel and drivers from a non-RAID disk, and once loaded, it can look in some table somewhere and see that it's actually supposed to boot from the array, and sort of re-boots to the array on the fly?

And when we say kernel above, maybe it doesn't necessarily mean OS kernel, but EFI kernel?

 

[nanofrog]

Ahhh, so maybe that's what I'm not understanding. Does that mean if I have a RAID array built with a 'bootable' RAID controller, the hardware still needs to at least load the kernel and drivers from a non-RAID disk, and once loaded, it can look in some table somewhere and see that it's actually supposed to boot from the array, and sort of re-boots to the array on the fly?

No.

The system is already pointed to the array for boot purposes (set in system firmware).

Generally speaking, the installation process is as follows (clean installation for a PC):
1. Install the RAID hardware (card, cables, and drives)
2. Set the system firmware (motherboard's settings)
3. Get into the RAID card's firmware and create the array (this can be done via a pre-existing OS, once the drivers have been installed to it, then ulimately, the OS location is moved to the RAID card)
4. Reboot, and begin the OS installation
5. Load the card's drivers when needed (and then the array shows up as an installation location) [adjusts the boot loader to use the card = system boots OS from the intended array]
6. Finish the OS install

Hopefully, the above will give you a good idea of what actually happens in the boot process, not to mention some insight into an OS installation, even on a MP (skip steps 2 & 3, as there's no access; read on to see what you do).

It's a little different on an MP, as there's no access to the system's firmware, nor can you get into the card's firmware by a keystroke method either (only works in BIOS based systems). Instead, you install the card in an existing version of OS X, and create the array, or place it in a PC and then transfer all the gear to the MP. The first way is easier IMO since you don't need to move hardware around.

And when we say kernel above, maybe it doesn't necessarily mean OS kernel, but EFI kernel?

It means OS kernel. EFI isn't a Kernel.

This might help.