Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

LSI1064 SAS

DearthnVader

DearthnVader

Suspended
Original poster
Dec 17, 2015
2,207
6,394
Red Springs, NC
I figured this SAS RAID card needed its own thread as you can pick them up for less than $20 USD and they work in all G5's out of the box. Either as a big 64 bit PCI card or a 64bit PCI-X card depending on if you Mac is equipped with PCI, or PCI-X.

I say it works out of the box because they have an FCODE ROM on them in addition to PC BIOS. However, there is not way to configure the advanced features of the card on the Mac OS as far as I know.

Mine works great as a 64bit PCI card with Apple's software RAID in Disk Utility( about 220MB's in striped RAID on two SATA III SSD's ) However in a PCI-X slot I am only seeing 269.551MB/s on the same RAID array.

These SSD each benchmark around 500MB/s in my x86 Linux box on the Mother Boards builtin SATA ports. The LSI card has 4 ports and each port should give 300MB/s on this card. I should be seeing 600Mb/s over two drives.

I don't know if the card needs to be configured with the LSI utility or what gives, surely the overhead of Apple's software RAID isn't 300MB/s+?????

Anyone got one of these cards in a PCI-X G5?
 
Amethyst1 said:
Did you configure the ports to negotiate 3 Gbps using lsiutil? They‘re configured to negotiate 1.5 Gbps IIRC — and 135 MB/s per port is what I would expect after overhead.
Click to expand...
That must be what it is , thanks. Doesn't look like there is a PowerPC binary but the Linux Makefile is just:

x
lsiutil: lsiutil.c
gcc -Wall -O lsiutil.c -o lsiutil
Click to expand...

So we should be able to make a version for the Mac OS.

Does the card itself have nvram settings that LSIutil saves to the card, so once we program it, it will save it's state if we swap it from a PC to a Mac?

Are you, or have you used the card in a PowetPC Mac?
 
I actually have the PCI-X version of the card, I have the PCI-E version too, but I don't think it will fit in a PCI slot?

So I'm going to have to cross compile the code for at least LinuxPPC to use the LSIUtil.

It should compile for the Mac OS, I don't see anything as to why it would not, just may take a little tweaking.

I wonder how it accesses the card, if drivers are needed for the Utility to be able to write to the NVRAM of the card or it's just user space access somehow and doesn't require a kernel extension to write to the NVRAM?

The Mac OS of course has a Kernel Extension for the card, but I don't know if it exposes the needed HW if the LSIUtil needs it.

We'll find out!
 
I think LSIUtil ignores any LSI driver and just accesses the card using PCI? In that case, you would create a Mac OS X target that uses the pciutils library to access the card.
Add defines to the gcc command for pciutils and macOS (make them separate in case another OS wants to use the pciutils method).
The gcc command should link the directhw library (make it static so other people don't need to install anything extra - they may need the DirectHW kext for old versions of Mac OS X such as for PPC).
gcc -DMACOSX -DPCIUTILS -Wall -O lsiutil.c -o lsiutil

flashrom is linked like this:
cc -o flashrom cli_classic.o cli_output.o cli_common.o print.o libflashrom.a -framework IOKit /usr/local/lib/libDirectHW.a -g -L/usr/local/lib -lpci -lresolv -framework CoreFoundation -framework IOKit -lz -lresolv -framework CoreFoundation -framework IOKit -lz -L/usr/local/Cellar/libusb/1.0.27/lib -lusb-1.0 -lobjc -Wl,-framework,IOKit -Wl,-framework,CoreFoundation -Wl,-framework,Security -L/usr/local/Cellar/libftdi/1.5_2/lib -L/usr/local/Cellar/libusb/1.0.27/lib -lftdi1 -lusb-1.0 -lobjc -Wl,-framework,IOKit -Wl,-framework,CoreFoundation -Wl,-framework,Security

So maybe you need to add /usr/local/lib/libDirectHW.a -g -L/usr/local/lib -lpci to the gcc command. This links the DirectHW library statically so that a user doesn't need to install the library (but they might need to load the DirectHW kext).

In the LSIUtil source code, you need to find every __linux__|__sparc__|__irix__|__alpha__|WIN32|DOS|EFI and decide how to add MACOS or PCIUTILS methods that do the same as whatever the block of code is doing.
Code: 
73    DOS
88    EFI
57    __linux__
56    __sparc__
54    WIN32
11    __irix__
 8    __alpha__

Before executing LSIUtil, you should maybe unload the kext for the card to be safe.
 
The card does fit in a 33MHz PCI 32bit slot, sadly it is not keyed for the 64bit PCI of the G4 Mac's, but I did test the PCI-E version of the card in a PCI to PCI-E bridge.

Something about the card or the bridge kills the watchdog timer of the G4's internal USB. In the QS you can't even boot OS X, in the MDD you can boot OS X but you won't have any input without USB.
 
DearthnVader said:
The card does fit in a 33MHz PCI 32bit slot, sadly it is not keyed for the 64bit PCI of the G4 Mac's, but I did test the PCI-E version of the card in a PCI to PCI-E bridge.

Something about the card or the bridge kills the watchdog timer of the G4's internal USB. In the QS you can't even boot OS X, in the MDD you can boot OS X but you won't have any input without USB.
Click to expand...
Maybe you can patch a kext to remove the watchdog timer? Is there an error message in the system log or console output? Maybe grep /System/Extensions for that error message string.
 
The OS X driver may set the link speed to 1.5.

I compiled the source code for LinuxPPC64 and set the link speed to 3.0. I confirmed it was still set to 3.0 after a few reboots, but my Max MB/s in QuickBenchX is still the same with my two drive RAID array.( 270MB/s ).


Booting back to Linux and checking the link speed the minimum is still set to 3.0.

More investigation to come.....
 
Setting up a single drive on port 3 with may RAID volume on port 1 and 2, the single drive gets over 220MB/s.

So we know it's a 3.0 link under OS X, just not sure why I don't see the gains I should on my striped RAID volume?

Only picking up 50MB/s or so from and striped RAID.......
 
DearthnVader said:
Setting up a single drive on port 3 with may RAID volume on port 1 and 2, the single drive gets over 220MB/s.

So we know it's a 3.0 link under OS X, just not sure why I don't see the gains I should on my striped RAID volume?

Only picking up 50MB/s or so from and striped RAID.......
Click to expand...
Does the card have a Hardware RAID option?

Is there a problem with the drives on port 1 and 2? Unraid them and check their individual bandwidth. If you're using software RAID, then you should be able to have one partition per disk that is for RAID and another partition per disk that is not RAID.

Does the PCI tab in System Information.app show anything different between PCI and PCI-X cards? In modern Macs, it shows PCIe link rate and link width.

Gather some PCI info from your Power Mac by loading or installing DirectHW, installing pciutils, and executing pcitree.sh and lspci -nnvvv
We can maybe see if it's negotiating PCI-X speed.

I'm working on a Mac OS X version of LSIUtil. It probably won't be useful but it might be interesting.
 
  • Like
Reactions: Amethyst1
joevt said:
Does the card have a Hardware RAID option?

Is there a problem with the drives on port 1 and 2? Unraid them and check their individual bandwidth. If you're using software RAID, then you should be able to have one partition per disk that is for RAID and another partition per disk that is not RAID.

Does the PCI tab in System Information.app show anything different between PCI and PCI-X cards? In modern Macs, it shows PCIe link rate and link width.

Gather some PCI info from your Power Mac by loading or installing DirectHW, installing pciutils, and executing pcitree.sh and lspci -nnvvv
We can maybe see if it's negotiating PCI-X speed.

I'm working on a Mac OS X version of LSIUtil. It probably won't be useful but it might be interesting.
Click to expand...
LSIUtil does have a section for RAID, but the manual I've found doesn't document it other than to say it is there. So I'll do some testing to see what it can do.

The documentation for the card does say each port is capable of 3.0, and that makes since because the card's OF name is LSI1064 as it is a 133MHz PCI-X card that's the max theoretical throughput of the bus.

Let me see if I can run lspci -nnvvv because ASP doesn't give any info other than normal PCI.

I'll look at the ioreg too and see what it says...
 
This is what Linux PPC reports for lspci -nnvvv

Code: 
0001:07:04.0 SCSI storage controller [0100]: Broadcom / LSI SAS1064 PCI-X Fusion-MPT SAS [1000:0050] (rev 02)
    Subsystem: Broadcom / LSI Device [1000:3020]
    Device tree node: /sys/firmware/devicetree/base/ht@0,f2000000/pci@2/LSILogic,sas@4
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
    Status: Cap+ 66MHz+ UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 248 (16000ns min, 2500ns max), Cache Line Size: 512 bytes
    Interrupt: pin A routed to IRQ 54
    Region 0: I/O ports at <ignored> [disabled] [size=256]
    Region 1: Memory at 90010000 (64-bit, non-prefetchable) [size=16K]
    Region 3: Memory at 90000000 (64-bit, non-prefetchable) [size=64K]
    Expansion ROM at 90200000 [disabled] [size=2M]
    Capabilities: [50] Power Management version 2
        Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
        Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [98] MSI: Enable- Count=1/1 Maskable- 64bit+
        Address: 0000000000000000  Data: 0000
    Capabilities: [68] PCI-X non-bridge device
        Command: DPERE- ERO- RBC=512 OST=16
        Status: Dev=07:04.0 64bit+ 133MHz+ SCD- USC- DC=simple DMMRBC=2048 DMOST=16 DMCRS=128 RSCEM- 266MHz- 533MHz-
    Capabilities: [b0] MSI-X: Enable- Count=1 Masked-
        Vector table: BAR=1 offset=00002000
        PBA: BAR=1 offset=00003000
    Kernel driver in use: mptsas
    Kernel modules: mptsas
 
I ran into a problem building DirectHW?

It doesn't build the .dmg and the kext it builds won't load.

Code: 
make -j2 directhw installmake -C DirectHW
make: *** No rule to make target `install'.  Stop.
make: *** Waiting for unfinished jobs....
xcodebuild -alltargets -project DirectHW10.5.xcodeproj 
cc DirectHW.c -dynamiclib -framework IOKit -o build/build10.5/Release/libDirectHW.dylib -g
In file included from DirectHW.c:19:
MacOSMacros.h:56:6: warning: #warning ppc
MacOSMacros.h:74:6: warning: #warning SDK 10.5
DirectHW.c: In function ‘MyIOConnectCallStructMethod’:
DirectHW.c:163: warning: ‘IOConnectMethodStructureIStructureO’ is deprecated (declared at /System/Library/Frameworks/IOKit.framework/Headers/IOKitLib.h:1314)
#cc -static -c DirectHW.c -o build/build10.5/Release/libDirectHW.a
=== BUILDING NATIVE TARGET KEXT OF PROJECT DirectHW10.5 WITH THE DEFAULT CONFIGURATION (Release) ===

Checking Dependencies...


=== BUILDING NATIVE TARGET DirectHW OF PROJECT DirectHW10.5 WITH THE DEFAULT CONFIGURATION (Release) ===

Checking Dependencies...


=== BUILDING NATIVE TARGET libDirectHW OF PROJECT DirectHW10.5 WITH THE DEFAULT CONFIGURATION (Release) ===

Checking Dependencies...
** BUILD SUCCEEDED **
james-lyonss-power-mac-g5:directhw-master jam$ sudo kextload -t /Users/jam/Documents/directhw-master/DirectHW/build/build10.5/Release/DirectHW.kext
kernel extension /Users/jam/Documents/directhw-master/DirectHW/build/build10.5/Release/DirectHW.kext has problems:
Authentication failures:
{
    "File owner/permissions are incorrect (must be root:wheel, nonwritable by group/other)" = (
        "/Users/jam/Documents/directhw-master/DirectHW/build/build10.5/Release/DirectHW.kext"
        "/Users/jam/Documents/directhw-master/DirectHW/build/build10.5/Release/DirectHW.kext/Contents/Info.plist"
        "/Users/jam/Documents/directhw-master/DirectHW/build/build10.5/Release/DirectHW.kext/Contents"
        "/Users/jam/Documents/directhw-master/DirectHW/build/build10.5/Release/DirectHW.kext/Contents/MacOS/DirectHW"
        "/Users/jam/Documents/directhw-master/DirectHW/build/build10.5/Release/DirectHW.kext/Contents/MacOS"
    )
}
Missing dependencies:
{
    "com.apple.kpi.mach" =
        "A valid compatible version of this dependency cannot be found"
    "com.apple.kpi.libkern" =
        "A valid compatible version of this dependency cannot be found"
}
 
joevt said:
I'm working on a Mac OS X version of LSIUtil. It probably won't be useful but it might be interesting.
Click to expand...
It would definitely be helpful, and likely useful if we can get it to work on the Mac OS.

Installing Linux is a pain for the "average user", tho I'm not sure how many "average users" we have left on PPC at this point.

It can be used on the PCI-E versions of the LSI SAS cards too, and these cards run $20 or less. If we can figure out how to get them up to full bandwidth we should be getting near 1GB/s on both cards as the PCI-E is 4x.

I mean those are unheard of speeds on PowerPC for the Mac OS.
 
  • Like
Reactions: AdamBuker
DearthnVader said:
If we can figure out how to get them up to full bandwidth we should be getting near 1GB/s on both cards as the PCI-E is 4x.
Click to expand...
800 MB/s on a Samsung SM951 AHCI SSD (PCIe ×4) is (probably) as good as it’s going to get. Since the SSD can do more, the host is the bottleneck.
 
Last edited:
  • Like
Reactions: AdamBuker
Amethyst1 said:
800 MB/s on a Samsung SM951 AHCI SSD (PCIe ×4) is (probably) as good as it’s going to get. Since the SSD can do more, the host is the bottleneck.
Click to expand...
Well PCI-E 1.0 does 1GB/s and so does PCI-X 133Mhz 64bit, and these drives are bootable.

On top of that AHCI SSD's are rare in sizes over 256GB, whereas SATA SSD's are $80 for 4TB and we can use 4 of them.

I mean you are "write" that 800MB/s is likely near the max actual throughput we would see real world, but that's nothing to laugh at.
 
joevt said:
Does the card have a Hardware RAID option?
Click to expand...
The LSIUtil does create RAID volumes, I made a mistake and RAIDed my boot drive.

Once I get OS X reinstalled we'll see if the volume I created in Stripe(0) shows up in the Disk Utility.....

I don't know if this card is true HW RAID, I doubt it, as those cards normally have RAM slots and a co-processor, tho one of the chips on the card does look like RAM, so who knows?

EDIT: The volume I created does show up as a single volume and Disk Utility was able to format it as a single volume.

Tests to come...
 
Last edited:
DearthnVader said:
The LSIUtil does create RAID volumes, I made a mistake and RAIDed my boot drive.

Once I get OS X reinstalled we'll see if the volume I created in Stripe(0) shows up in the Disk Utility.....

I don't know if this card is true HW RAID, I doubt it, as those cards normally have RAM slots and a co-processor, tho one of the chips on the card does look like RAM, so who knows?

EDIT: The volume I created does show up as a single volume and Disk Utility was able to format it as a single volume.

Tests to come...
Click to expand...
Oops, got all excited and wiped my Linux drive......

The volume I created with LSIUtil does NOT show up in Disk Utility at all.....
 
DearthnVader said:
It doesn't build the .dmg and the kext it builds won't load.
Click to expand...
I updated my DirectHW fork. I had to make a change to my Mac OS X 10.3 compatibility commit to make sure Mac OS X 10.4 and later still work.

Amethyst1 said:
800 MB/s on a Samsung SM951 AHCI SSD (PCIe ×4) is (probably) as good as it’s going to get. Since the SSD can do more, the host is the bottleneck.
Click to expand...
The host has an x16 slot so you can remove the x4 bottleneck by using a PCIe gen 3 x16 bridge. One of the PCIe 3.0 x16 bridges that I have (Netstor NA255A) works in my B&W G3 and Power Mac 8600 but that is limited by the PCI slot to ≈1 Gbps.

I haven't tried the Netstor in the Quad G5 yet but I did try the Amfeltec Squid PCIe Gen 3 Carrier Board for 4 M.2 SSD modules and also the HighPoint SSD7505. One of those didn't work well (I don't remember which). I didn't test any PCIe devices connected to the M.2 slot (I only verified that the NVMe devices could be detected). The HighPoint SSD7505 is actually gen 4 which isn't useful in a gen 1 x16 host unless you have a gen 4 device that is x2 or x1.
 
  • Like
Reactions: Amethyst1
I think I have this thing almost sorted. I did not realize I had 2 240GB SATA6 drives and one 256GB SATA6 drive.

All are the same brand and should be operating at 3.0Gb/s but the two 240 drives are somehow setting to 1.5Gb/s.

It maybe cabling, it maybe just these two drives don't want to play nice with the LSI card.

I have all four ports set to 3.0GB/s in LSIUtil.

I'll order some new high quality cables and report back.

The 256GB drive is hitting 270MB/s on it's own in both Linux and the Mac OS so we know it is good for 3.0Gb/s and about what the overhead is.

The two 240GB drives RAIDed together with the LSIUtil show up under linux and I can format and test them, and they hit 270MB/s in RAID 0.

They don't show up at all under the Mac OS, not even in diskutil list.

But if I SW RAID them with OS X Disk Utility they still hit 270MB/s in RAID 0.
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back