EGPU enclosure for 2018 Mac Mini?

[bzgnyc2]

Can someone still clarify how USB3, NVMe, video, and graphics data routes around the Mac Mini 2018 as far as the TB3 and HDMI?

From this thread still not quite clear how this works in the base (no external GPU) configuration:
-Does video from the Intel Graphics out through the HDMI count towards any bandwidth limits elsewhere?

-Does video from the Intel Graphics out through one of the TB3 ports reduce the available bandwidth on the other TB3 port of that bus?
+If so, how much bandwidth of TB3's theoretical 40Gbps would remain for NVMe?
+If the display is operating at a resolution using 16Gbps bandwidth, does that still leave 40-16=24Gbps (22 after overhead) for the NVMe on the same bus? Or is it 4x8 PCIe serial lines = 32Gbps - 16Gbps = 16Gbps for the NVMe?
+Either way, is any reduction symmetric or would writes to the NVMe be reduced more than reads if the display is using most of the outgoing bandwidth but presumably none of the incoming (I understand TB3 to be full-duplex)?

-If I have a USB3 device and a NVMe device on the same Thunderbolt bus, could I potentially get 10 + 22 Gbps across the two of them or would the USB3 device's bandwidth come out of the 22 Gbps (after overhead) available for PCIe devices on TB3?

-Can the two Thunderbolt buses run at full bandwidth simultaneouosly or are the other internal limits (e.g. total PCIe bandwidth on which the Intel controllers sit) that limit total bandwidth across them to less than 2x32?

-Does the situation/rules change if the other device on the same Thunderbolt bus is an eGPU rather an NVMe?

 

[padams35]

Can someone still clarify how USB3, NVMe, video, and graphics data routes around the Mac Mini 2018 as far as the TB3 and HDMI?

From this thread still not quite clear how this works in the base (no external GPU) configuration:
-Does video from the Intel Graphics out through the HDMI count towards any bandwidth limits elsewhere?

-Does video from the Intel Graphics out through one of the TB3 ports reduce the available bandwidth on the other TB3 port of that bus?
+If so, how much bandwidth of TB3's theoretical 40Gbps would remain for NVMe?
+If the display is operating at a resolution using 16Gbps bandwidth, does that still leave 40-16=24Gbps (22 after overhead) for the NVMe on the same bus? Or is it 4x8 PCIe serial lines = 32Gbps - 16Gbps = 16Gbps for the NVMe?
+Either way, is any reduction symmetric or would writes to the NVMe be reduced more than reads if the display is using most of the outgoing bandwidth but presumably none of the incoming (I understand TB3 to be full-duplex)?

-If I have a USB3 device and a NVMe device on the same Thunderbolt bus, could I potentially get 10 + 22 Gbps across the two of them or would the USB3 device's bandwidth come out of the 22 Gbps (after overhead) available for PCIe devices on TB3?

-Can the two Thunderbolt buses run at full bandwidth simultaneouosly or are the other internal limits (e.g. total PCIe bandwidth on which the Intel controllers sit) that limit total bandwidth across them to less than 2x32?

-Does the situation/rules change if the other device on the same Thunderbolt bus is an eGPU rather an NVMe?

1. Each Thunderbolt Bus has 32Gbps of Data (PCIe 3.0 x4)
1b - There appears to be 10Gbps of data reserved for each port, leaving 22Gbps maximum one any one port.
2. The Thunderbolt Busses share 2x DisplayPort 1.2 channels usable on either bus (In addition to the PCIe data)
2b - The Mini's UHD 630 (not Iris) integrated graphics supports 3x DisplayPort 1.2 outputs, with the third displayport channel hard tied to the HDMI port.
3. The maximum Data + Video on any one port is 40Gbps.

To answer your questions:
+ No, HDMI output shouldn't count toward any bandwidth limits.
+ 16Gbps of DisplayPort data should not cause any NVMe reduction. 40 Gpbs - 16Gpbs DisplayPort still leaves 24 Gbps which is greater than the 22Gbps practical data limit per port.
+ Correct, you should be able to run 10Gbps USB-C on one port with a 22Gbps Thunderbolt-3 on the second port of the same bus.
+ Each Thunderbolt bus should have a separate pool of PCIe 3.0x4 data.
+ USB3 + NVMeSSD works just as well as USB3 + eGPU (but try not to put a Thunderbolt3 NVMe SSD on the same bus as a Thunderbolt3 eGPU).

Edit: This is just bandwidth theory. Some overhead/latency may apply in practice.

 

[PaulD-UK]

@bzgnyc2 "...would writes to the NVMe be reduced more than reads if the display is using most of the outgoing bandwidth but presumably none of the incoming..."

Speaking generally (not just for an Intel Mac mini), then yes. It is often the case that SSD Write speed over TB3 is reduced if an alt-mode DP display is connected to the same TB port.
In practice this is quite likely to be a greater reduction than the theoretical calculation would suggest...

 

[joevt]

1. Each Thunderbolt Bus has 32Gbps of Data (PCIe 3.0 x4)
+ Correct, you should be able to run 10Gbps USB-C on one port with a 22Gbps Thunderbolt-3 on the second port of the same bus.
+ Each Thunderbolt bus should have a separate pool of PCIe 3.0x4 data.

You can get ≈22 Gbps of data per port, but only ≈23 Gbps total for two ports of the same Thunderbolt controller instead of the expected ≈28 Gbps. Of course 32 Gbps total is impossible since 8*4*128/130 = 31.5 Gbps, but also overhead, thus 28 Gbps is a conservative maximum for PCIe 3.0 x4 (like what you would see from a NVMe in a PCIe slot of a Mac Pro 2019). I don't know why a Thunderbolt controller can only deliver ≈23 Gbps of that.

You can test multiple ports simultaneously using ATTO Disk Benchmark.app without having to create a RAID since it allows selecting multiple drives at once.

 

[padams35]

You can get ≈22 Gbps of data per port, but only ≈23 Gbps total for two ports of the same Thunderbolt controller instead of the expected ≈28 Gbps. Of course 32 Gbps total is impossible since 8*4*128/130 = 31.5 Gbps, but also overhead, thus 28 Gbps is a conservative maximum for PCIe 3.0 x4 (like what you would see from a NVMe in a PCIe slot of a Mac Pro 2019). I don't know why a Thunderbolt controller can only deliver ≈23 Gbps of that.

You can test multiple ports simultaneously using ATTO Disk Benchmark.app without having to create a RAID since it allows selecting multiple drives at once.

This can’t be right. I don’t have any Nvme Thunderbolt SSDs so I can’t disprove with a drive test, but I’ll see if I can get CL!ng to provide numbers for parking an eGPU and a USB 3.1 drive on the same bus later this week.

------
Edit: Huh... preliminary results not as expected. CL!ng says my eGPU only runs at 15.5-16.5 Gbps Host to Device; 21.1-22.9 Gbps Device to Host regardless of what is/isn't connected to the bus. I feel like I might be missing bandwidth. Meanwhile Black Magic pegged my SATA SSD as 460 MB/s write / 514 MB/s read, or about +3.7 Gbps out / +4.1 Gbps in when plugged into a USB-C port on the same bus.

~16 Gbps out + ~4 Gbps out = <22Gbps and proves nothing.
~22 Gbps In + ~4 Gbps In would be greater than 22Gbps... if CL!ng didn't show eGPU Device to Host bandwidth dropping to ~20 Gbps at regular intervals that might match up to the read tests.

 

[joevt]

~16 Gbps out + ~4 Gbps out = <22Gbps and proves nothing.

True. You need faster devices - at least with a total possible bandwidth to exceed the 31.5 Gbps of PCIe 3.0 gen 4 to test the limits of a discrete Thunderbolt host controller.
I used two 22 Gbps NVMe drives. The total did not get very far beyond 23 Gbps.

~22 Gbps In + ~4 Gbps In would be greater than 22Gbps... if CL!ng didn't show eGPU Device to Host bandwidth dropping to ~20 Gbps at regular intervals that might match up to the read tests.

You can't run two different apps and add their bandwidth together to get a total because you can't be sure that one app is not sending while the other is idle which would inflate the total.
You need to connect max seed SSDs to each port and use a benchmark that reads or writes to multiple devices at the same time, like ATTO Disk Benchmark.app.