Specifics of M3 Pro dual external monitor limitations

[sammko]

TLDR: What exactly is the reason for M3 Pro dual external monitor capabilities being lower than those of M3 Max? There are 2 DCPs routed to the crossbar, I am assuming the DCPs are the same as in M3 Max, but yet, there is a difference. Is it a bandwidth limitation of the crossbar?

I have two high-bandwidth monitors hooked up to a Thunderbolt 4 dock:

1. 4K 240Hz HBR3 with DSC
2. 1440p 165Hz HBR2 (macOS only lists 120Hz for this one)

I have two machines available for testing this setup: M3 Pro MacBook Pro and M3 Max MacBook Pro.

To start off, almost every source lists that 4K 240Hz is only supported over HDMI. This is not true, it works just fine via USB-C as well, whether DP 4-lane alt-mode or tunnelled over TB4. As far as I know the HDMI port is converted from HBR3 internally anyway, so can't go over 32 Gbps to reach the full 48 Gbps of HDMI2.1.

M3 Max​

The M3 Max supports this configuration with no problems. Both monitors work over a single TB4 connection. The specific configuration it picks is HBR3 for the 4K monitor and HBR for the 1440p using DSC. I think not using DSC for the 1440p screen wouldn't quite fit into the 40 Gbps limit. ioreg output in this state:

"AppleTypeCPhyDisplayPortTunnel" = {"Tunnel 0"={"Link Rate"="8.10Gbps/lane (HBR3)","Client"="AppleATCDPINAdapterPort(atc2-dpin0)"},"Tunnel 1"={"Link Rate"="2.70Gbps/lane (HBR)","Client"="AppleATCDPINAdapterPort(atc2-dpin1)"}}

Sadly this depends on the order in which the screens are detected. If the 1440p is first, it gets HBR2 and then the 4K gets HBR2 as well, dropping down to 4K 120Hz. If the 4K is detected first, it's fine. Oh well.

The M3 Max is happy doing HBR3+HBR2 as well, as long as they are not on the same port.

M3 Pro​

The M3 Pro cannot do HBR3+HBR. If the 4K screen is detected first, it gets an HBR3 stream and the second one never lights up. This is also the case if it plugged in to a different port on the machine, so not a Thunderbolt issue. HBR2+HBR2 works the same as for the M3 Max.
Why is this? Is this a crossbar limitation?

When HBR3 is active and a second screen is connected (but not on), it can be seen under pending-dfps:

+-o AppleT603XDisplayCrossbar(display-crossbar0)  <class AppleT603XDisplayCrossbar, id 0x10000040c, registered, matched, active, busy 0 (1 ms), retain 30>
    {
      "IOClass" = "AppleT603XDisplayCrossbar"
      "CFBundleIdentifier" = "com.apple.driver.AppleDisplayCrossbar"
      "IOProviderClass" = "AppleARMIODevice"
      "supportsDualPipe" = Yes
      "available-ufps" = ()
      "IOProbeScore" = 100000
      "IONameMatch" = "display-crossbar,t603x"
      "IOMatchedAtBoot" = Yes
      "IOMatchCategory" = "IODefaultMatchCategory"
      "EventLog" = ...
      "dfp-endpoints" = ("atc0","atc1","atc2","atc3","lpdptx")
      "IOPersonalityPublisher" = "com.apple.driver.AppleDisplayCrossbar"
      "IONameMatched" = "display-crossbar,t603x"
      "CFBundleIdentifierKernel" = "com.apple.driver.AppleDisplayCrossbar"
      "ufp-endpoints" = ("dispext0","dispext1","dispext2","dispext3","scodec0")
      "current-state" = {"dfp"=("0:ufp0,0-dfp2,1"),"ufp"=("0:ufp1,0-ufp0,0")}
      "pending-dfps" = ("atc1,0")
    }

 

[sammko]

Digging deeper into this on the M3 Pro, the HBR combination is a red herring. It in fact can do HBR3+HBR on a single Thunderbolt port, as well as HBR3 + HDMI (@HBR3).

Proof (HBR3 + HDMI@HBR3):

| |   |       "AppleTypeCPhyDisplayPortPclk" = {"PCLK 1"={"Clients"=["AppleATCDPINAdapterPort(atc1-dpin0)"],"Link Rate"="8.10Gbps/lane (HBR3)"}}
| |   |       "AppleTypeCPhyDisplayPortPclk" = {"PCLK 1"={"Clients"=["AppleATCDPHDMIPort(atc3-dpphy)"],"Link Rate"="8.10Gbps/lane (HBR3)"}}

Sorry for lack of context, this is grepped output of a single ioreg invocation with a HBR3 monitor plugged in over Thunderbolt (atc1-dpin0) and another in HDMI (atc3-dpphy).

Proof (HBR3 + HBR on same port):

| |   |       "AppleTypeCPhyDisplayPortPclk" = {"PCLK 1"={"Clients"=["AppleATCDPINAdapterPort(atc1-dpin0)"],"Link Rate"="8.10Gbps/lane (HBR3)"},"PCLK 2"={"Clients"=["AppleATCDPINAdapterPort(atc1-dpin1)"],"Link Rate"="2.70Gbps/lane (HBR)"}}
    | |   |       "AppleTypeCPhyDisplayPortTunnel" = {"Tunnel 0"={"Link Rate"="8.10Gbps/lane (HBR3)","Client"="AppleATCDPINAdapterPort(atc1-dpin0)"},"Tunnel 1"={"Link Rate"="2.70Gbps/lane (HBR)","Client"="AppleATCDPINAdapterPort(atc1-dpin1)"}}

So we can rule out a crossbar limitation causing this. Which would have been surprising anyway, as the crossbar is not the expensive (silicon-wise) component of this whole thing, rather the DCPs are.

---

Now how did we get here and why does the M3 Pro typically refuse to get into this configuration?

I'm guessing each DCP has a max pixel clock limitation that's enough for 4K@120, but not 4K@240. Not sure if pixel clock explicitly, but it's not resolution or refresh rate (it can do 4K@120 or 1080p@240 on its own). It turns out the 4K@240 mode is driven by both DCPs in "dual pipe" mode. Each of them produces 1920x2160@240Hz and they're combined.
I'm not joking! Some logs:

default    00:19:26.390651+0000    kernel    [DCPEXT0:PPipeDCP_H13P.cpp:22630] Dual pipe enabled offset 0
default    00:19:26.390656+0000    kernel    [DCPEXT0:PPipeDCP_H13P.cpp:22631] VFTG merge enabled
default    00:19:26.390661+0000    kernel    [DCPEXT0:PPipeDCP_H13P.cpp:22632] VFTG primary
default    00:19:26.591768+0000    kernel    [DCPEXT0:eoInterfaceIOAV.cpp:783] Merged!! Active 3840 hbp 80 hfp 48 hs 32
default    00:19:26.601614+0000    kernel    [DCPEXT0:nifiedPipeline.cpp:9574] IOMFBStatus UnifiedPipeline::mode_set_gated(uint32_t, uint32_t): 1920x2160@239 Hz DSC=YES VRR=NO link: 1

default    00:19:26.610121+0000    kernel    [DCPEXT1:PPipeDCP_H13P.cpp:22630] Dual pipe enabled offset 1
default    00:19:26.610126+0000    kernel    [DCPEXT1:PPipeDCP_H13P.cpp:22631] VFTG merge enabled
default    00:19:26.610127+0000    kernel    [DCPEXT1:PPipeDCP_H13P.cpp:22632] VFTG secondary
default    00:19:26.812746+0000    kernel    [DCPEXT1:nifiedPipeline.cpp:9574] IOMFBStatus UnifiedPipeline::mode_set_gated(uint32_t, uint32_t): 1920x2160@239 Hz DSC=YES VRR=NO link: 1

Notice dcpext0 reporting "VFTG primary" and dcpext1 reporting "VFTG secondary". It is unclear where the signals are combined, whether it's a display-crossbar0 functionality (IMO not likely), or the dcpext0 receives the signal from dcpext1 (IMO more likely) and combines them.

The crossbar configuration hints at this, also visible in the OP, but I did not understand what it meant:
"current-state" = {"dfp"=("0:ufp0,0-dfp1,1"),"ufp"=("0:ufp1,0-ufp0,0")}
ufp0,0 connected to dfp1,1 (the atc1-dpin0 dpxbar input) and ufp1,0 connected to ufp0,0 on the "ufp" side. Whatever that actually means in practice.

The way to get the HBR3+HBR configuration is to first plug in a monitor over HDMI (or any other port than our 1st TB4), the plug in HBR3 into our TB4 port. This makes sure only one DCP is used to drive the HBR3 screen. Next unplug HDMI and plug in another monitor to the TB4 port.

---

To conclude, a single DCP cannot drive a 4K@240 screen. They seem to be limited by a pixel clock (or equivalent) that is enough for 6K@60 (so somewhere around 1500 MHz) but not enough for 8K@60 or 4K@240 (around 2000 MHz). This holds for the M3 Max too, but it has 4 DCPs to play around with so can handle both monitors using 3 DCPs.

Plugging in two AW3225QF's to the M3 Max (DP and HDMI) (ok it's the same monitor connected twice, I only have one), we get this gem:
"current-state" = {"dfp"=("0:ufp0,0-dfp2,1","0:ufp2,0-dfp3,0"),"ufp"=("0:ufp1,0-ufp0,0","0:ufp3,0-ufp2,0")}
ufp1+ufp0 handling dfp2,1 (DP) and ufp3+ufp2 handling dfp3,0 (HDMI)