SAS card for G5Quad

[socom_22]

So I ended up ordering a couple of those HP-branded LSI SAS3041E-HP cards. I figured that if the chipset is identical, they should be flashable.

Upon receiving the cards and installing them, sure enough, they prevented my Quad from booting - frozen at the Apple logo. No worries, immediately into a DOS PC they went and I flashed them using the process consolidated by @V.Yakob in post 203. I then used LSIUtil to configure the cards to run at 3-gigabit speeds.

This is to confirm that this worked for me with the HP cards. As you can see, OpenFirmware detects the card as a scsi-2 device and the FCode expansion ROM has been successfully loaded. I did not test booting because I really don't want to disassemble my FC setup right now, but I am quite confident that the cards are bootable.

 

[Tratkazir_the_1st]

OpenFirmware detects the card as a scsi-2 device

This is really funny thing . I think so - if card has FCode ROM onboard & you see it's properties in OF Shell - it have to be bootable.
BTW - you can try to find LSI3081E-R - 8 port elder brother, SFF-8087 ports . According to official specs - it supports SPARC Solaris & obviously have FCode ROM onboard. https://docs.broadcom.com/doc/12352186

 

[socom_22]

This is really funny thing . I think so - if card has FCode ROM onboard & you see it's properties in OF Shell - it have to be bootable.

It's actually completely normal and expected. SAS = Serial Attached SCSI and the LSI SAS1064E "Fusion MPT" chip is essentially just a multi-port SCSI host bus controller.

We also cannot assume that just because the Fcode has loaded properly and the card is configured with additional properties on the OF device tree, that the card is bootable. The necessary drivers must be implemented to allow booting. For LSI SAS and FC cards with Fcode ROMs, usually they are (for OS X at least).

This is the reason why no one has figured out how to boot OS9 from LSI fibre channel or SAS cards yet. OS9 does include drivers/extensions for these types of controllers but sadly cannot boot from them.

 

[socom_22]

As a follow-up to my previous post about flashing the HP-branded cards, I discovered an interesting limitation with the G5 Quad which I also encountered previously with other LSI fibre channel cards as well.

What I discovered a couple years ago is that if I take any 3 LSI Logic fibre channel cards and plug them into a Quad, it works just fine and I can boot normally. If I add a fourth card, the system does not boot into OS X but instead redirects to the OpenFirmware prompt showing the below error:

Method <SetQuietEnable> not found; ihandle=0 phandle=0

The interesting part is that ANY combination of >3 LSI Logic Fusion-MPT controllers (any mix of >3 SAS or Fibre Channel LSI cards present) will refuse to boot.

I have been stumped by this limitation for years now and I need some help to figure it out.

I need to understand what exactly this error means and what might be preventing multiple FC/SAS controllers from working together. As I mentioned any combination of 3 controllers or less present on the PCI bus work just fine. I am not clear whether this is a limitation in OpenFirmware or whether it is a limitation or conflict in the LSI Logic firmware which initializes and configures the Fusion-MPT controllers. If anyone has any ideas I am all ears.

If I can find a way around this limitation, it would allow me to use use more controllers simultaneously and unlock some really amazing storage possibilities never before attempted on a PowerPC Mac.

 

[Tratkazir_the_1st]

This is the reason why no one has figured out how to boot OS9 from LSI fibre channel or SAS cards yet.

By "bootable" I mean that LSI controller/HBA installs it's PCI ROM & export drives further, as it works with PCs. SCSI\SAS HBAs usually load PCI ROM & install their BIOS extension, and drives are visible through int13h (excuse me if I explain it cumbersome ). After successful BIOS install drives accessible even in DOS . BTW, where can I read about similar procedures but for PowerMacs?

 

[joevt]

As a follow-up to my previous post about flashing the HP-branded cards, I discovered an interesting limitation with the G5 Quad which I also encountered previously with other LSI fibre channel cards as well.

What I discovered a couple years ago is that if I take any 3 LSI Logic fibre channel cards and plug them into a Quad, it works just fine and I can boot normally. If I add a fourth card, the system does not boot into OS X but instead redirects to the OpenFirmware prompt showing the below error:

Method <SetQuietEnable> not found; ihandle=0 phandle=0

The interesting part is that ANY combination of >3 LSI Logic Fusion-MPT controllers (any mix of >3 SAS or Fibre Channel LSI cards present) will refuse to boot.

I have been stumped by this limitation for years now and I need some help to figure it out.

I need to understand what exactly this error means and what might be preventing multiple FC/SAS controllers from working together. As I mentioned any combination of 3 controllers or less present on the PCI bus work just fine. I am not clear whether this is a limitation in OpenFirmware or whether it is a limitation or conflict in the LSI Logic firmware which initializes and configures the Fusion-MPT controllers. If anyone has any ideas I am all ears.

If I can find a way around this limitation, it would allow me to use use more controllers simultaneously and unlock some really amazing storage possibilities never before attempted on a PowerPC Mac.

[B]Method <...> not found; ihandle=0 phandle=0[/B]
Is an error message from $call-method which has existed since the B&W G3.

Can you show .properties and words from each card? Or just do a dump-device-tree to capture everything.

Does .log show anything interesting?

If you have a second Mac or PC, then you can capture text from Open Firmware using telnet.

Maybe there's a limit in Open Firmware that is being reached?

There might be some ideas for debugging at https://forums.macrumors.com/thread...l-work-in-a-beige-power-macintosh-g3.2303689/

 

[flyproductions]

The necessary drivers must be implemented to allow booting. For LSI SAS and FC cards with Fcode ROMs, usually they are (for OS X at least).

Yes, looks like they are! The Quad, Sorbet Leopard in this case, boots perfectly fine with the card. Even a Raid0 which the Mac Pro cannot.

 

[tfelder]

Now I'm in search for SAS card (or at least SATA2) for G4 (MDD) .

So... a bit late to reply here, but you can definitely do this with the SAS3041E cards.

• It only works with 1 disk attached, probably because the controller refuses to initialize more than a single PCI lane when it's on the PCI-PCIe-Adapter.
• It's not faster than an IDE-to-SATA adapter

 

[Tratkazir_the_1st]

Overkill.

 

[Betonmischer_86]

I have the exact same LSI SAS3041E-R, and it booted in my G5 Quad out of the box. Until I decided to flash it from IR to IT mode to disable the built-in RAID capabilities and gain some extra performance that way. As expected, the latest version of bios from Broadcom made the system unbootable until I applied the fcode patch from this thread. Curiously, when I reverted both firmware and bios from backup to those the card came with, it won't boot either. Would someone take a look at my original bios dump to figure out why?

 

[willowthewisp]

I have the exact same LSI SAS3041E-R, and it booted in my G5 Quad out of the box. Until I decided to flash it from IR to IT mode to disable the built-in RAID capabilities and gain some extra performance that way. As expected, the latest version of bios from Broadcom made the system unbootable until I applied the fcode patch from this thread. Curiously, when I reverted both firmware and bios from backup to those the card came with, it won't boot either. Would someone take a look at my original bios dump to figure out why?

Put the hba in a PCIe based PC and check the entries in the cards NVRAM (bios) ! To my knowledge the fcode reads the cards NVRAM too ! If the NVRAM is corrupted or has inappropriate boot settings the g5 will see the ssd but won’t boot from it.

Just to make sure if I did get it right - is the g5 not booting with the hba inserted (no Apple logo) or is the g5 just deneying to boot from the attached ssd ?

 

[Betonmischer_86]

Put the hba in a PCIe based PC and check the entries in the cards NVRAM (bios) ! To my knowledge the fcode reads the cards NVRAM too ! If the NVRAM is corrupted or has inappropriate boot settings the g5 will see the ssd but won’t boot from it.

To be clear, that's what did work so far and what didn't:

1. My HBA with its BIOS, firmware and NVRAM settings as it came out of the box was bootable in the G5.
2. I nuked the HBA with 'sasflash -o -e 6' to install the latest BIOS and IT mode firmware from Broadcom. Otherwise, it wouldn't let me write an IT mode firmware over an IR one. After that the HBA was no longer bootable. The G5 would still boot from other sources (internal SATA etc.).
3. I erased the HBA once more and restored the original BIOS and firmware from backup, but that didn't make it bootable.
3. The FCODE patch from this thread over the latest IT firmware and BIOS from Broadcom made it bootable again. Which is how I currently use it.

Here's what I make out of this. The original BIOS had FCODE included along with other boot services, as the HBA was bootable in the G5 out of the box. But FCODE alone apparently isn't enough, you need specific NVRAM variables to be set. On the other hand, if all other boot services except for FCODE are removed (which is what the patch does), the default NVRAM variables work just fine.

If we can figure out what those variables are, for SOME cards there will be no need for a patched BIOS, provided the stock one has FCODE included. I don't mind using a custom BIOS, but modifying this old HBA is a huge pain. From what I found, you need a PC motherboard with either a BIOS (not UEFI) or a 32-bit UEFI to be able to erase the flash memory with sasflash. I'm not sure if the same is true for straight upgrading though, as I didn't try to do that in a modern PC.

Unfortunately, I didn't take note of the original variables in the PC boot utility (or lsutil) prior to nuking the flash. And I'm not eager to set up a Socket AM2 bench system again for experimentation, to be honest. Still, I'm willing to give it a try someday.

BTW, switching from IR to IT firmware seems to be worth it in the end. My 4KB sequential reads in Xbench went from 270 to 290 MB/s using two 15K HDDs in RAID0. That could be due to a newer firmware alone though. I wonder if anyone else would like to try it and post their results. Apparently, those HBAs are set up for IR by default.

 

[joevt]

LSIUtil might work in Mac OS X. https://forums.macrumors.com/threads/lsi1064-sas.2429351/post-33245146

 

[itanium_hunter]

Does anyone know whether it's possible to get an ATTO ExpressSAS 6Gb card bootable in a PCIE G5?

There's an open firmware rom file dated August 2008 in some of the older firmware bundles. I've tried with an H608 but haven't had much luck - I'm not much of an PPC Mac expert though.

 

[joevt]

Does anyone know whether it's possible to get an ATTO ExpressSAS 6Gb card bootable in a PCIE G5?

There's an open firmware rom file dated August 2008 in some of the older firmware bundles. I've tried with an H608 but haven't had much luck - I'm not much of an PPC Mac expert though.

That Open Firmware ROM is only 512 bytes. It only sets some properties. It has no code for reading blocks so it can't be used for booting.

 

[Betonmischer_86]

It looks like there may indeed be no need to remove the PC boot service by patching the ROM to make the Fcode boot work. On a Windows machine, I used the official Broadcom tool to flash the latest IT mode firmware and BIOS, then used lsiutil's option 61 to revert non-default settings and option 13 to set the minimum link speed to 3gbps. After that, the PC boot settings menu was back and the adapter was still bootable on the Mac.

 

[Betonmischer_86]

I wonder if something can be done to speed up the boot process, though. In my G5 Quad, it takes a good minute for the SAS drives to begin spinning up, even though I didn't touch the default spin-up delay settings (2 s). Correct me if I'm wrong, but SCSI/SAS drives don't have their own spin-up delay settings, which could be the reason. Is there, perhaps, a predetermined boot order in OpenFirmware that makes the Mac look for other boot options before using external sources (which the PCIe HBA is one of)?

 

[joevt]

It looks like there may indeed be no need to remove the PC boot service by patching the ROM to make the Fcode boot work. On a Windows machine, I used the official Broadcom tool to flash the latest IT mode firmware and BIOS, then used lsiutil's option 61 to revert non-default settings and option 13 to set the minimum link speed to 3gbps. After that, the PC boot settings menu was back and the adapter was still bootable on the Mac.

The PCI option ROM that I can see in that Broadcom tool download (SAS3041ER_ Package_P21_IR_IT_Firmware_BIOS_for_MSDOS_Windows) does not have an Open Firmware PCI option ROM image. It has 4 LSI ROM images after the BIOS image. The BIOS image is marked as the last image even though it is the first (since the other images are not EFI/BIOS/Open Firmware firmware boot images?)
The .fw files appear to be a different kind of firmware than the PCI option ROM.

If that PCI option ROM was flashed to the card then it wouldn't be bootable for Power PC Mac.

I wonder if something can be done to speed up the boot process, though. In my G5 Quad, it takes a good minute for the SAS drives to begin spinning up, even though I didn't touch the default spin-up delay settings (2 s). Correct me if I'm wrong, but SCSI/SAS drives don't have their own spin-up delay settings, which could be the reason. Is there, perhaps, a predetermined boot order in OpenFirmware that makes the Mac look for other boot options before using external sources (which the PCIe HBA is one of)?

Does the G5 Quad have a correct device and file path set in the boot-device nvram variable so that it doesn't need to search for a boot device?
Use printenv and dump-device-tree to find out.

go to 68kmla.org and search the forums for multi-boot and multi-boot-menu to learn about the boot picker in New World Power Macs.

 

[Betonmischer_86]

The PCI option ROM that I can see in that Broadcom tool download (SAS3041ER_ Package_P21_IR_IT_Firmware_BIOS_for_MSDOS_Windows) does not have an Open Firmware PCI option ROM image. It has 4 LSI ROM images after the BIOS image. The BIOS image is marked as the last image even though it is the first (since the other images are not EFI/BIOS/Open Firmware firmware boot images?)
The .fw files appear to be a different kind of firmware than the PCI option ROM.

If that PCI option ROM was flashed to the card then it wouldn't be bootable for Power PC Mac.

Now that I think of it, the ROM contents I ended up with might not be entirely stock. The whole flashing cycle for me was like this:

1. Erase the firmware and ROM that came from the factory.
2. Write the recent firmware and ROM from the Broadcom site.
3. Write the fcode ROM patch. -> HBA bootable in the Mac, but not in a PC.
4. Write the ROM from Broadcom once again without erasing anything first. -> HBA bootable in both environments.

Does the G5 Quad have a correct device and file path set in the boot-device nvram variable so that it doesn't need to search for a boot device?

Thank you. The Mac now boot right into the SAS array after:
setenv boot-device /ht/pci@6/LSILogic,sas@0/disk@0:3,\\:tbxi

 

[Tratkazir_the_1st]

Write the fcode ROM patch.

Where can I found this one? I've got Fcode ROM for SAS2, but couldn't find for SAS1 cards...

 

[Betonmischer_86]

Where can I found this one? I've got Fcode ROM for SAS2, but couldn't find for SAS1 cards...

I used the patched ROM from #203

 

[Betonmischer_86]

@joevt Also, what tools did you use to look for boot services in the ROM?

 

[joevt]

@joevt Also, what tools did you use to look for boot services in the ROM?

My DumpPCIRom.sh script dumps the different image in the PCI option ROM.
https://github.com/joevt/OpenBIOSStuff
https://forums.macrumors.com/thread...l-work-in-a-beige-power-macintosh-g3.2303689/

A PCI Option ROM contains 'PCIR' near the beginning.
The .fw files do not contain that - they do not contain a PCI Option ROM header.

 

[Betonmischer_86]

@joevt Thank you. I fed my current ROM to the script and got the following output. This ROM is the result of flashing the latest official ROM from Broadcom to an erased HBA, then flashing the Fcode patch over it and flashing the official ROM again. The HBA is currently bootable on both Mac and PC with the latest IT mode firmware.

user@home-lpt-1 ~ % ./DumpPCIRom.sh -l bios_current.rom
========================
DumpPCIRom.sh log:

# Working Path: /Users/user
# Script Name: ./DumpPCIRom.sh
# Script Path: .
# TempFolder: /tmp/DumpPCIRom.IJ3sLS
# Doing part: 0
# Got PCI Header
# start:0 offset:68
# part:0 type:"BIOS" indicator:"another image" offset:0 size:b200
# Doing part: 1
# Got PCI Header
./DumpPCIRom.sh: line 229: $((0x${PointerPCIDataStructure} - 28))*2: substring expression < 0
# start:b200 offset:b232
# part:1 type:"Open Firmware" indicator:"last image" offset:b200 size:7200
# Doing part: 2
# Got PCI Header
# start:12400 offset:12434
# part:2 type:"??" indicator:"another image" offset:12400 size:a00
# Doing part: 3
# Got PCI Header
# start:12e00 offset:12e34
# part:3 type:"??" indicator:"another image" offset:12e00 size:1ca00
# Doing part: 4
# Got PCI Header
# start:2f800 offset:2f834
# part:4 type:"??" indicator:"another image" offset:2f800 size:7000
# Doing part: 5
# Got PCI Header
# start:36800 offset:36834
# part:5 type:"??" indicator:"another image" offset:36800 size:400

I also found the backup of the ROM my HBA came with and fed it to the script too:

user@home-lpt-1 ~ % ./DumpPCIRom.sh -l bios_factory.rom
========================
DumpPCIRom.sh log:

# Working Path: /Users/user
# Script Name: ./DumpPCIRom.sh
# Script Path: .
# TempFolder: /tmp/DumpPCIRom.ww3F6m
# Doing part: 0
# Got PCI Header
# start:0 offset:68
# part:0 type:"BIOS" indicator:"another image" offset:0 size:b200
# Doing part: 1
# Got PCI Header
./DumpPCIRom.sh: line 229: $((0x${PointerPCIDataStructure} - 28))*2: substring expression < 0
# start:b200 offset:b232
# part:1 type:"Open Firmware" indicator:"last image" offset:b200 size:7200
# Doing part: 2
# Got PCI Header
# start:12400 offset:12434
# part:2 type:"??" indicator:"another image" offset:12400 size:a00
# Doing part: 3
# Got PCI Header
# start:12e00 offset:12e34
# part:3 type:"??" indicator:"another image" offset:12e00 size:1c600
# Doing part: 4
# Got PCI Header
# start:2f400 offset:2f434
# part:4 type:"??" indicator:"another image" offset:2f400 size:7000
# Doing part: 5
# Got PCI Header
# start:36400 offset:36434
# part:5 type:"??" indicator:"another image" offset:36400 size:400

The difference between the two is that the factory ROM is an older version and will no longer allow booting on Mac for some reason (even with the matching firmware) after the HBA has been erased before (could be due to some specific NVRAM settings).

I'm attaching all the files in an archive for comparison:

bios_factory.rom — the ROM my adapter came with
bios_latest_official.rom — the latest ROM from the Broadcom website
bios_patched.rom — modified ROM with only Fcode in boot services
bios_current.rom — the ROM I currently use, which is bootable on PC and Mac alike

Only one question remains — with the janky way the current ROM is produced (flashing official->patched->official), is it put together correctly? If not, is it possible to do the same thing without mangling the ROM?

There's also the error in the script on line 229.

 

[Betonmischer_86]

By the way, sasflash (attached) runs on PPC Linux. I used a fork of Void on my Power Mac G5 to dump the ROM. Only low-level (DOS, EFI) versions allow erasing any part of the HBA flash, though, which is necessary to switch from IR to IT mode firmware.