Darwin is open source - isn't that useful for something?

[randomshinichi]

Apple has sources for Darwin all the way back to the very first versions of OS X. I was thinking, that's got to be useful for something, right?

Apple Open Source

opensource.apple.com

For example:
1. looking at the changes between 10.6 and 10.5 kernel, at least one could get a 10.6 or later kernel to boot on PPC.
2. at the very least, one could definitely write a NVMe driver for Powermac G5s?
3. maybe this could help inform Linux PPC/QEMUdevelopment? For example, sleep doesn't work on Linux PPC (at least for my Powermac G5), but there is plenty of stuff referencing hibernation

But surely one must have thought of all this before. What don't I know? Or maybe nobody just had the time to understand all that?

 

[joevt]

Apple has sources for Darwin all the way back to the very first versions of OS X. I was thinking, that's got to be useful for something, right?

Apple Open Source

opensource.apple.com

For example:
1. looking at the changes between 10.6 and 10.5 kernel, at least one could get a 10.6 or later kernel to boot on PPC.
2. at the very least, one could definitely write a NVMe driver for Powermac G5s?
3. maybe this could help inform Linux PPC/QEMUdevelopment? For example, sleep doesn't work on Linux PPC (at least for my Powermac G5), but there is plenty of stuff referencing hibernation

But surely one must have thought of all this before. What don't I know? Or maybe nobody just had the time to understand all that?

It was useful for getting 10.4 to work on earlier PowerPC Macs like the Power Mac 8500/8600.
https://eshop.macsales.com/OSXCenter/XPostFacto/Framework.cfm?page=XPostFacto3.html

Darwin does not include source code for many drivers (Thunderbolt, USB, NVMe, GPUs, ...). Some code that used to be included is not updated.

A USB XHCI driver for macOS was created but is not updated. It may still work though and would be useful for getting USB 3.2 gen 2x2 (20 Gbps) speeds.
https://github.com/search?q=GenericUSBXHCI&type=repositories
https://forums.macrumors.com/threads/usb-3-x-pcie-cards-for-classic-mac-pro.1501482/

An NVMe driver (NVMeGeneric.kext) for macOS was created but the source code was not released.
http://www.macvidcards.com/blog/nvme-driver-now-available-download-and-enjoy
Another person would have to recreate that work. There's a NVMe website that has source code. There's NVMe code in Linux and edk2 on git hub.
https://nvmexpress.org/drivers/

Using those sources, creating an NVMe kext for PowerPC Macs shouldn't be too difficult. You could probably even boot from it by porting the driver to Open Firmware.

 

[randomshinichi]

An NVMe driver (NVMeGeneric.kext) for macOS was created but the source code was not released.
http://www.macvidcards.com/blog/nvme-driver-now-available-download-and-enjoy
Another person would have to recreate that work. There's a NVMe website that has source code. There's NVMe code in Linux and edk2 on git hub.
https://nvmexpress.org/drivers/

Using those sources, creating an NVMe kext for PowerPC Macs shouldn't be too difficult. You could probably even boot from it by porting the driver to Open Firmware.

1. The NVMe driver: is this related to the NVMe driver that you can flash onto a cMP 3,1 to have it boot from NVMe? or is this something totally different?
2. Yes I'd be interested in a NVMe kext for PowerPC OS X, but what would porting a driver to Open Firmware look like? I don't need the source code for the Powermac's Boot ROM?

 

[Kainrzog]

There are some people that are interested in the open-source aspect of Darwin, like in the PureDarwin project:

Welcome to PureDarwin

Moving the Darwin community in the right direction!

www.puredarwin.org

You might want to ask your questions there.

 

[joevt]

1. The NVMe driver: is this related to the NVMe driver that you can flash onto a cMP 3,1 to have it boot from NVMe? or is this something totally different?
2. Yes I'd be interested in a NVMe kext for PowerPC OS X, but what would porting a driver to Open Firmware look like? I don't need the source code for the Powermac's Boot ROM?

1. It's a kext called NVMeGeneric.kext for macOS. It was created for macOS Sierra before macOS had proper support for third party NVMe. The NVMe driver that can be flashed or loaded from disk is a driver for EFI (required for boot before macOS is loaded).
2. You would write it in Forth, or maybe it can be compiled from C ( https://www.openfirmware.info/SmartFirmware ). It would have the same kind of methods that any block storage device would have. Section 3.7.2 of the "IEEE Standard for Boot (Initialization Configuration) Firmware: Core Requirements and Practices" document coredoc/1275-1994/1275.pdf) describes block devices. You would need a method to load this driver - maybe from a supported disk or the EPROM of a PCIe device (flash it on an unused GPU). The Open Firmware environment should be enough for testing the driver (like the UEFI Shell).

Most of the contents of the PowerMac's boot ROM is obtainable - it can be dumped and disassembled and detokenized. The tokenized parts can be converted to Forth. I have done this with a few Macs including the Quad G5.

Power Mac Quad G5 should be able to handle GPT formatted disks
https://forums.macrumors.com/threads/booting-a-guid-disk-on-powerpc-macs.2113689/

You might not need an Open Firmware driver. It may be sufficient to have the macOS driver on a booting helper volume. The helper volume loads some early macOS drivers including disk drivers like the NVMe driver. Those disk drivers allow access to the disk containing macOS. Control is passed to that disk. It's like the Preboot volume on modern macOS.
https://forums.macrumors.com/threads/boot-leopard-from-pcie-ssd-on-g5-quad.2183549/

 

[randomshinichi]

Great, because I do have a Quad G5. OK so let me see if I understand this correctly.

There is a C/C++/ObjC driver out there called NVMeGeneric.kext to make OS X work with NVMe drives (Why can't I find it at https://opensource.apple.com/release/macos-1012.html, it seems very hard to find the actual source code for this?). One could get this to compile with a 10.4/10.5 kernel.. You could put it onto a partition that the firmware normally supports, so that the kernel can use it to find a root partition on a NVMe disk. This is like Linux's initrd on /boot on a separate partition.

Alternately one could dump the Powermac's boot ROM, disassemble it (it is in Forth), edit it, and flash it back (not going to put this on a separate PCIe card). How does one do this? So I'd have to write a NVMe driver in Forth, or compile from C down into Forth. How would you test such a driver without flashing the ROM back to the Powermac each time? How capable is this "Open Firmware environment" - if I'm developing on a Linux machine, wouldn't it need direct hardware access to a NVMe drive, bypassing Linux's NVMe driver, to test anything?

 

[Amethyst1]

You could alternatively stick to PCIe AHCI SSDs (which 10.4.11 sadly doesn’t seem to like) and forego NVMe…

 

[randomshinichi]

You could alternatively stick to PCIe AHCI SSDs (which 10.4.11 sadly doesn’t seem to like) and forego NVMe…

That's not quite the point - I want to make something cool on the hardware I own, not use something already out there.

 

[Amethyst1]

That's not quite the point - I want to make something cool on the hardware I own, not use something already out there.

Understood. I was just saying that (some) PCIe SSDs can already be used with PPC Macs/early versions of OS X.

 

[DearthnVader]

How would you test such a driver without flashing the ROM back to the Powermac each time?

The same way you test any driver for a PCI device, you get the current instance and load the driver from disk and test/debug it.

First you have to port the driver to forth then tokenize it.

dev /pci/@f <--get the PCI NVME device
" /pci/@f" open-dev to my-self <- Get the current instance
load hd:4,\ppc\nvme.fcode <--Load the fcode rom
800000 1 byte-load <--execute the fcode from the loadbase address in ram

 

[joevt]

There is a C/C++/ObjC driver out there called NVMeGeneric.kext to make OS X work with NVMe drives (Why can't I find it at https://opensource.apple.com/release/macos-1012.html, it seems very hard to find the actual source code for this?).

I said it wasn't made by Apple and that the source code was not available. The kext can be downloaded from hackintosh websites, but it's probably Intel only and only supports Sierra and later.

One could get this to compile with a 10.4/10.5 kernel.. You could put it onto a partition that the firmware normally supports, so that the kernel can use it to find a root partition on a NVMe disk. This is like Linux's initrd on /boot on a separate partition.

Right. I think it's called boot≠root in macOS.
https://developer.apple.com/library...eptual/KernelProgramming/booting/booting.html

Alternately one could dump the Powermac's boot ROM, disassemble it (it is in Forth), edit it, and flash it back (not going to put this on a separate PCIe card). How does one do this?

It's easy to get the ROM from earlier PowerMacs. I think for Quad G5 Power Macs, I could only get the compiled FCode because the ROM is compressed or encrypted or something. The ROM in Quad G5 Power Macs is nicer because it has more features and all the functions are named (not anonymous) with fcode-debug? set to true.

So I'd have to write a NVMe driver in Forth, or compile from C down into Forth.

I think the edk2 driver might be the best candidate for porting, at least for the Open Firmware driver - fewer source files to look at.

How would you test such a driver without flashing the ROM back to the Powermac each time? How capable is this "Open Firmware environment" - if I'm developing on a Linux machine, wouldn't it need direct hardware access to a NVMe drive, bypassing Linux's NVMe driver, to test anything?

Direct hardware access - do you mean in a VM? I think that's a possibility in Linux VMs. However the Open Firmware in those VMs (Probably QEMU with OpenBIOS?) is not exactly the same as the Open Firmware of a Power Mac and I don't know if such a VM (with Open Firmware) has direct access. I suppose what you would want to do is have a development environment to compile the Fcode, then have a method to transfer that Fcode to a Power Mac's Open Firmware environment. Open Firmware can use telnet for the console. I'm not sure it supports hot plug of USB or FireWire devices (probably not). In that case you need to restart for each test (probably should do that anyway to reset the test environment anyway).

You could alternatively stick to PCIe AHCI SSDs (which 10.4.11 sadly doesn’t seem to like) and forego NVMe…

Does Open Firmware on a Power Mac Quad G5 support AHCI PCIe? Probably not. But if Leopard supports AHCI, then it would be interesting to use this to test boot≠root. I would try this first. Then do boot≠root for your NVMe kext. Then do Open Firmware after that.

The same way you test any driver for a PCI device, you get the current instance and load the driver from disk and test/debug it.

First you have to port the driver to forth than tokenize it.

dev /pci/@f <--get the PCI NVME device
" /pci/@f" open-dev to my-self <- Get the current instance
load hd:4,\ppc\nvme.fcode <--Load the fcode rom
800000 1 byte-load <--execute the fcode from the loadbase address in ram

fcode-utils can tokenize. My own fcode script/tools can convert tokenized fcode back to forth. So you can test tokenization with a round trip: forth -> tokenize -> back to forth and compare.
https://forums.macrumors.com/threads/imac-g3-tray-bootrom-dump-1mbyte.2316567/post-30429064
I've never tried the SmartFirmware C to Forth/Fcode compiler.

I don't have a tool to convert compiled FCode to Forth. But if Open Firmware's see command can do it (for words that don't include assembly only opcodes) then it could be possible. Power Mac Quad G5 also has see-words (see-words) see-word xsee (xsee) but I don't know what they do. The compiled Fcode may give a clue.