Mac Mini PCIE lanes and TB3

astrocramp

macrumors member
Original poster
Aug 27, 2011
67
39
My i7 Mini is on the way, hopefully here by Friday to start crunching data. I have a question on Thunderbolt 3 and the Mac as it relates to PCIE lanes. From what I understand, each TB3 port has bandwidth equivalent to 4x PCIe lanes.

Basically, I want to hookup my Mini to 2x Dell U2719DC USB-C monitors via 2x TB3 to the monitors' USB-C, which would drive the displays and also light up the USB3 ports on the monitors. A third port will eventually be used for a RAID array (I might do this via the 10Gbps NIC I configured it with, TBD). I have to believe that a few of those TB3 ports are direct to CPU, and a few share PCIE lanes via DMI with the USB ports, Bluetooth, NIC, etc.

There doesn't seem to be a wealth of info on the Mac platform and how I/O is apportioned to the limited number of lanes. I never cared about this topic on Mac, since I used only Mac laptops in light duty and my Windows workstation for productivity. This mini will be in workstation duty pretty much all day with a decent amount of peripherals - I want to be sure I'm not unnecessarily crippling I/O to the display or other onboard I/O by using shared lanes. Yes, I could test it, but having the documentation / info would be better. I had a look at the Mini user manual, and there's no reference to this.
 

treekram

macrumors 68000
Nov 9, 2015
1,849
401
Honolulu HI
According to this article, the Thunderbolt controllers (2 of them, 2 ports each) are connected directly to the CPU with 4 PCIe lanes each. (So each controller with 2 ports gets the 4 PCIe lanes.)
https://egpu.io/forums/desktop-computing/mac-mini-review-heedlessly-renovated/

If you plug in a DisplayPort monitor such as the Dell U2719DC, what probably happens is that the DisplayPort output from the CPU/iGPU is routed through the Thunderbolt controller to the USB-C port. The Intel CPU's with the integrated GPU's have dedicated DisplayPort lanes from the CPU/iGPU. According to the Intel documentation, the HDMI output from the CPU is 4096x2304@30Hz HDMI 1.4 which tells me that the HDMI port on the Mini is probably DisplayPort output converted to HDMI since the HDMI port on the Mini is 4096x2160@60Hz HDMI 2.0.

If you have a RAID device which has PCIe NVMe SSD's, to me, it wouldn't make sense to connect that using a 10Gbps NIC if it also had a TB3 port because the TB3 port is 40Gbps and a single capable NVMe SSD should be able to do more than 10Gbps.

From what you describe, 2 QHD monitors with a single RAID device is very (actually very, very) comfortably within the limits of what the Mini can handle from the IO standpoint. The DisplayPort output comes directly from the iGPU/CPU and if the article is correct, with the amount of PCIe lanes the Thunderbolt ports get, there really shouldn't be a problem there. The external USB ports probably get their output from the PCH CM246 (according to iFixit) connected by DMI to the CPU. But unless you plan to have multiple USB SSD's or peripherals with similar bandwidth constantly pushing data, I don't see a problem with your USB devices getting throttled either.
 
  • Like
Reactions: astrocramp

astrocramp

macrumors member
Original poster
Aug 27, 2011
67
39
According to this article, the Thunderbolt controllers (2 of them, 2 ports each) are connected directly to the CPU with 4 PCIe lanes each. (So each controller with 2 ports gets the 4 PCIe lanes.)
https://egpu.io/forums/desktop-computing/mac-mini-review-heedlessly-renovated/

If you plug in a DisplayPort monitor such as the Dell U2719DC, what probably happens is that the DisplayPort output from the CPU/iGPU is routed through the Thunderbolt controller to the USB-C port. The Intel CPU's with the integrated GPU's have dedicated DisplayPort lanes from the CPU/iGPU. According to the Intel documentation, the HDMI output from the CPU is 4096x2304@30Hz HDMI 1.4 which tells me that the HDMI port on the Mini is probably DisplayPort output converted to HDMI since the HDMI port on the Mini is 4096x2160@60Hz HDMI 2.0.

If you have a RAID device which has PCIe NVMe SSD's, to me, it wouldn't make sense to connect that using a 10Gbps NIC if it also had a TB3 port because the TB3 port is 40Gbps and a single capable NVMe SSD should be able to do more than 10Gbps.

From what you describe, 2 QHD monitors with a single RAID device is very (actually very, very) comfortably within the limits of what the Mini can handle from the IO standpoint. The DisplayPort output comes directly from the iGPU/CPU and if the article is correct, with the amount of PCIe lanes the Thunderbolt ports get, there really shouldn't be a problem there. The external USB ports probably get their output from the PCH CM246 (according to iFixit) connected by DMI to the CPU. But unless you plan to have multiple USB SSD's or peripherals with similar bandwidth constantly pushing data, I don't see a problem with your USB devices getting throttled either.
Thank you! This forum delivers. I was mostly concerned that the TB3 ports were sharing a single controller or worse going through DMI. It sounds like the Mini is a standard 8th gen chip with 20 lanes of PCIe - 8x to the TB3 ports, 4x to the NVMe, 4x to 10Gbps NIC (I read that on another link), and 4x to chipset DMI (USB ports, wireless, Bluetooth, 1Gbps NIC).

In that case, it probably would make sense to connect an eGPU or (iGPU to display) to one controller, and then RAID to the other controller so that they are using 8x PCIe lanes rather than sharing 4x on one controller. I suppose I can dig into system configuration menus to see which TB3 port is which.

Interesting side note after reading up on all this - the 10Gbps NIC must be soldered to the board otherwise we would have heard about an open PCIe slot (which would be perfect for user added NVMe).
 

treekram

macrumors 68000
Nov 9, 2015
1,849
401
Honolulu HI

rmdeluca

macrumors 6502
Oct 30, 2018
250
392
So how the remaining 8 lanes are shared between NVMe, 10Gbps NIC and chipset DMI ????
DMI is separate from the PCIe lanes. It’s equivalent to a x4 PCIe 3.0 connection on Coffee Lake CPUs. The NVMe and 10GbE are on PCIe lanes.
 

dia3olik

macrumors member
Sep 14, 2007
83
18
Italy
DMI is separate from the PCIe lanes. It’s equivalent to a x4 PCIe 3.0 connection on Coffee Lake CPUs. The NVMe and 10GbE are on PCIe lanes.
So 10GbE shares 4GB/s of bw with the USB3.1 Gen2 ports and wireless and BT, right???