Does AppleSoc support IOMMU for directed IO?

[DearthnVader]

So little documentation on PCI Passthrough on ARACH64.

I can't seem to get vfio-pci to bind to a device.

[    0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-linux-asahi root=UUID=1977b9ca-42c7-4d1e-b414-abd00f14503b rw loglevel=3 quiet vfio_iommu_type1.allow_unsafe_interrupts=1
[    0.364284] vfio_pci: add [14e4:4425[ffffffff:ffffffff]] class 0x000000/00000000
[    0.364287] vfio_pci: add [14e4:5f69[ffffffff:ffffffff]] class 0x000000/00000000
[    0.651983] vfio-pci 0000:01:00.0: Adding to iommu group 3
[    0.675375] vfio-pci: probe of 0000:01:00.0 failed with error -22
[    0.675414] vfio-pci 0000:01:00.1: Adding to iommu group 3
[    0.699367] vfio-pci: probe of 0000:01:00.1 failed with error -22

 

[DearthnVader]

What do you make of this @joevt?

sudo dmesg | grep iommu
[    0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-linux-asahi root=UUID=1977b9ca-42c7-4d1e-b414-abd00f14503b rw loglevel=3 quiet vfio_iommu_type1.allow_unsafe_interrupts=1
[    0.015862] iommu: Default domain type: Translated
[    0.015863] iommu: DMA domain TLB invalidation policy: strict mode
[    0.392309] apple-dart 24e808000.iommu: DART [pagesize 4000, 16 streams, bypass support: 1, bypass forced: 0] initialized
[    0.392453] apple-dart 382f00000.iommu: DART [pagesize 4000, 16 streams, bypass support: 1, bypass forced: 0] initialized
[    0.392535] apple-dart 382f80000.iommu: DART [pagesize 4000, 16 streams, bypass support: 1, bypass forced: 0] initialized
[    0.392673] apple-dart 502f00000.iommu: DART [pagesize 4000, 16 streams, bypass support: 1, bypass forced: 0] initialized
[    0.392761] apple-dart 502f80000.iommu: DART [pagesize 4000, 16 streams, bypass support: 1, bypass forced: 0] initialized
[    0.392956] apple-dart 235004000.iommu: DART [pagesize 4000, 16 streams, bypass support: 1, bypass forced: 0] initialized
[    0.402836] dockchannel-hid 24eb30000.input: Adding to iommu group 0
[    0.414640] rtkit-helper 24e400000.mtp: Adding to iommu group 0
[    0.454369] dwc3 382280000.usb: Adding to iommu group 1
[    0.476370] dwc3 502280000.usb: Adding to iommu group 2
[    0.651768] OF: /soc/pcie@690000000: no iommu-map translation for id 0x0 on (null)
[    0.651983] vfio-pci 0000:01:00.0: Adding to iommu group 3
[    0.675414] vfio-pci 0000:01:00.1: Adding to iommu group 3
[    1.123954] apple-admac 238200000.dma-controller: Adding to iommu group 4

 

[joevt]

What do you make of this @joevt?

I've never researched or attempted pci passthrough.

 

[PowerPC74]

It is possible to connect any PCIe device, but it is not possible to use eGPUs with standard software/games without (possibly very slow) workarounds due to technicalities involving memory mapping modes. This is not a firmware limitation, it is a limitation of the Apple Silicon bus fabric / memory subsystem itself which cannot support externally attached memory over PCIe that works like system memory does (and eGPUs have VRAM which software expects to access as normal RAM).

macOS fully supports Thunderbolt with that limitation too, you're just going to have a hard time finding Apple Silicon-compatible drivers for most hardware like that on macOS. This isn't a problem on Linux, everything that already has a driver in mainline Linux should be supported except GPUs.

There is an existing kernel workaround for this in some kernel trees and we'll probably ship it for ***** and giggles at some point (though there is next to no chance it will ever be upstreamed), but I'm not holding my breath that it won't slow down many GPU workloads to a crawl. Maybe not though.

https://www.reddit.com/r/AsahiLinux/comments/132nxv4/does_apple_silicon_support_pcie_34_over

So it's more a limitation of UMA+bus fabric on AppeSoC than it is firmware or security features, so that just leaves the AppleSoC Mac Pro and it's own PCI-E slots and how those may work or not work with IOMMU and GPU VRAM?

Seems to me they would also be limited by the UMA+bus fabric, no?

I also doubt Apple is going to address this with M5+, it's a basic limitation of the SoC.

 

[PowerPC74]

To me without PCI Passthrough there is no reason to run Linux, basically everything you can do on Linux and be done on the macOS on AppleSoc.

The other consideration is Linux can not be booted form an external device on AppleSoC, you could likely chroot into a Linux install on an external device but that still requires the root file system on the internal SSD and that comes with some issues later if you ever want to get rid of it. Apple's Disk Unity does not handle it and even the installer on a USB can't repartition the drive and get rid of the linux install.

What I had to do is restore the M2 MacBook Pro with Apple Configurator with another Mac with USB-C, just Lucky I had one because my Linux install was taking up about 64GB of the 256GB SSD and it was not use to me.