Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

shaocaholica

macrumors regular
Original poster
May 26, 2010
128
31
I haven't been able to find any good write ups on how TB3/4 works when it goes to a dock and the dock has multiple output devices.

For this example lets assume a TB4 MacBook Pro connected to a TB4 dock and the dock has everything a TB4 dock can have as outputs.

1) Is there a single thunderbolt signal from the computer to the dock or are there multiple multiplexed signals?

2) Is a Thunderbolt4 signal the same as USB4 signal or are they completely different at the logical signal level? If USB devices are present on the dock are their signals repacked into the thunderbolt signal or are they just multiplexed?

3) What if (multiple) DisplayPort signals are passed to the dock? Are they sent as packets within the Tb4 signal or are they pure DP signal that's multiplexed with the TB4 signal?

4) What if the MacBook or source computer has a discrete GPU passing a DP signal down to the dock, how is that signal mixed with the other USB and Tb signals?

Basically it seems like on the computer side there can be multiple devices that need to get data down that single cable and on the dock side there can be multiple devices that take the data. How is this managed?
 
1) Is there a single thunderbolt signal from the computer to the dock or are there multiple multiplexed signals?
One Thunderbolt signal. It may contain tunnelled data for one PCIe device (the upstream PCI bridge of the Thunderbolt dock), one or two DisplayPort devices, and one USB device (the upstream connector of a USB hub but only if the host is Thunderbolt4 or USB4 and the dock is Thunderbolt 4 or USB4).

2) Is a Thunderbolt4 signal the same as USB4 signal or are they completely different at the logical signal level? If USB devices are present on the dock are their signals repacked into the thunderbolt signal or are they just multiplexed?
Thunderbolt 4 is similar to USB4 but not exactly the same. A USB4 device might connect at 40 Gbps instead of Thunderbolt's 41.25Gbps.

A Thunderbolt 4 or USB4 host will use USB tunnelled over Thunderbolt to communicate with a USB hub in the Thunderbolt 4 or USB4 controller of a dock.

A Thunderbolt 1,2,3 dock does not support tunnelled USB. A Thunderbolt 1,2,3 host cannot tunnel USB. In either case, the host (Thunderbolt or USB4) will use PCIe tunnelled over Thunderbolt to communicate with a PCIe USB Controller in the Thunderbolt or USB4 controller of the dock.

3) What if (multiple) DisplayPort signals are passed to the dock? Are they sent as packets within the Tb4 signal or are they pure DP signal that's multiplexed with the TB4 signal?
Packets. Like how Ethernet can send HTTP, FTP, etc.

4) What if the MacBook or source computer has a discrete GPU passing a DP signal down to the dock, how is that signal mixed with the other USB and Tb signals?
The Discrete GPU (AMD or Intel) has two DisplayPort outputs connected to two DisplayPort inputs of a Thunderbolt controller so that the Thunderbolt controller can tunnel the DisplayPort data to the Thunderbolt controller of a dock or display which will convert the tunnelled DisplayPort data back to DisplayPort.

Basically it seems like on the computer side there can be multiple devices that need to get data down that single cable and on the dock side there can be multiple devices that take the data. How is this managed?
Similar to how networks can send data from different source to different destinations over the same cable.

PCIe is like a network. The Thunderbolt dock looks like a PCIe bridge - there's an upstream PCIe bridge and a downstream bridge for each PCIe device in the dock and for each downstream Thunderbolt port. PCIe

USB is like a network. In the case where tunnelled USB is supported, the Thunderbolt 4 or USB4 dock acts like a USB hub - there's an upstream USB port and 4 downstream USB ports for an internal USB connection and the three downstream Thunderbolt 4 or USB4 ports.

DisplayPort is not like a network - but the Thunderbolt topology is. There's a Thunderbolt path that is setup between the DisplayPort In Adapter of the host and the DisplayPort Out Adapter of the dock or display. Actually DisplayPort MST is like a network but Thunderbolt doesn't deal with MST - it doesn't behave like an MST hub.

A Thunderbolt or USB4 controller has adapters to handle converting USB, PCIe, DisplayPort to/from Thunderbolt data. Use IORegistryExplorer.app to see these adapters in macOS.
- DP In Adapter on the host. DP Out Adapter on Thunderbolt docks/displays/Adapters. The BlackMagic eGPUs and Sonnet eGPU Breakaway Puck 5500 XT/5700 also have Dp In Adapters.
- USB Down Adapter for downstream and USB Up Adapter for upstream.
- PCIe Down Adapter for downstream and PCIe Up Adapter for upstream.

Read the USB4 spec for all the details. The latest version is 2.0 which includes 80 Gbps mode and an asymmetrical 120 Gbps/40 Gbps mode equivalent to Thunderbolt 5.
 
Thanks for that detailed write up.

So on TB4 ports that also support pure USB and DP, the TB4 chip does some handshake and switching allowing the upstream USB/DP device to connect to the appropriate onboard component.

And USB-C Power Delivery is independent of the signal type? The flow of power is independent of the signal type? A display can draw power or give power. A TB dock can draw power or give power. A USB device can draw power or give power.
 
So on TB4 ports that also support pure USB and DP, the TB4 chip does some handshake and switching allowing the upstream USB/DP device to connect to the appropriate onboard component.
All Thunderbolt 4 ports support pure USB and DP.

The work of setting up the DisplayPort path from the In Adapter to the Out Adapter is done automatically by the host Thunderbolt controller's Thunderbolt Internal Connection Manager or it could be done by the host Thunderbolt driver (Thunderbolt External Connection Manager) which is usually the case with macOS.

In iMacs with Thunderbolt 1, the host Thunderbolt controller contains a DisplayPort Out Adapter which you don't see in later host Thunderbolt controllers. A Mac connected with Thunderbolt to the Thunderbolt 1 iMac can setup a cross domain path between the DisplayPort In Adapter of its host Thunderbolt controller to the DisplayPort Out Adapter of the iMac's host Thunderbolt controller so that the Mac can use the iMac's display as if it were connected to the Mac's GPU. Pressing Command-F2 on the iMac triggers the process of setting up the path. It first switches the iMac's display from the iMac's GPU to the DisplayPort Out Adapter of the iMac's host Thunderbolt controller. This entire process is called Thunderbolt Target Display Mode.

The iMac also somehow accepts audio from the DisplayPort signal but the iMac has to use some driver to extract the audio ?somehow? to send it to the iMac's speakers. I don't know how that's done.

And USB-C Power Delivery is independent of the signal type? The flow of power is independent of the signal type? A display can draw power or give power. A TB dock can draw power or give power. A USB device can draw power or give power.
I think so.

Doesn't seem to be available. Github .dmg hangs on "Preparing to download".
Go to https://developer.apple.com/download/all/
Search for hardware tools. IORegistryExplorer.app is included in download packages like "Additional Tools for Xcode 15" back to 2016, and "Hardware IO Tools" before that.

Or you can use the ioreg command like this:
ioreg | grep Thunderbolt
 
Thanks again.

So then running a 6k display over TB3/4 will greatly reduce the available bandwidth for other devices that are also piggybacking on that same TB connection. I guess this would be the case with the 6k Studio Display
 
Using a TB3 ( 3X4 JHL7440) dock with an enclosed WD SN750 2TB SSD, connected to an M1 Mac mini:
No other connections to the dock gives 2700-2800MB read/write.

If I connect a 5k 10 bit 27" monitor (USB-C to DP 1.4 cable with DSC) to the dock I get:
2700MB/s read and 1300MB/s write.

So the inbound TB data to the computer remains the same, but write data bandwidth out of the computer to the disk is shared with the monitor.
 
So then running a 6k display over TB3/4 will greatly reduce the available bandwidth for other devices that are also piggybacking on that same TB connection. I guess this would be the case with the 6k Studio Display
The Apple Pro Display XDR has two methods of receiving 6K60.

1) Dual HBR3 over Thunderbolt which takes 38.9 Gbps. This is a special mode for DisplayPort 1.4 capable GPUs that don't support DSC. Apple allows this mode only if there is no Thunderbolt device between the host and the display. This mode can only be setup by software (Aople's External Connection Manager) because dual HBR3 51.84 Gbps would normally exceed Thunderbolt bandwidth. The reason it works is because each 3008x3384 30bpp 60Hz half of the display does not require the entire bandwidth of HBR3 x4 and Thunderbolt does not transmit the stuffing symbols that DisplayPort uses to fill up the bandwidth. This mode probably doesn't work in Linux or Windows (unless they've been updated to use an External Connection Manager that can detect this situation). I think USB4 doesn't have an Internal Connection Manager? But I don't know if Linux or Windows External Connection Managers know how to allow dual HBR3.
https://www.kernel.org/doc/html/latest/admin-guide/thunderbolt.html
https://learn.microsoft.com/en-us/w...t-guidelines/usb4-intro-to-connection-manager

2) HBR2 with DSC. This mode takes 15.4 Gbps which is less bandwidth than 4K60 (16 Gbps).

DisplayPort is mostly one direction. This means only transmit/write speed is affected. Receive/read speed is not greatly affected. I have not seen anyone test the read/write speed of a USB device connected to an XDR that is using dual HBR3 mode.

Data is usually limited to ≈ 22 Gbps (≈25 Gbps max) so a 6K60 DSC display or 4K60 display won't affect write speed much.

Using a TB3 ( 3X4 JHL7440) dock with an enclosed WD SN750 2TB SSD, connected to an M1 Mac mini:
No other connections to the dock gives 2700-2800MB read/write.

If I connect a 5k 10 bit 27" monitor (USB-C to DP 1.4 cable with DSC) to the dock I get:
2700MB/s read and 1300MB/s write.

So the inbound TB data to the computer remains the same, but write data bandwidth out of the computer to the disk is shared with the monitor.
5K60 DSC should be even less bandwidth (12 Gbps) than 6K60 DSC. A drop in write speed to 1300MB/s is greater than I would expect.
Which 5K display? Are you sure it's using DSC?
5K60 can connect with HBR3 without DSC at 8bpc (22 Gbps).
Usually a DSC connection uses 12bpc and compresses the stream to an average of 12bpp.

The script at https://gist.github.com/joevt/e862b0088ef58b9144877d01401bcee8 will list modes on an Apple Silicon Mac and it will indicate the current mode.
 
I'm using a DIY 5K 27" display with an iMac Pro screen (new) and a Chinese R1811 v4 video driver board, set to DP 1.4 mode.
So its technically an 8 bit + FRC screen?
Connected by a USB-C>DP 1.4 cable (CableMatters 8K).

The current mode seems to be:
-> 5120x2880@59.937Hz 175.316kHz 925.67MHz h(48 32 80 +) v(3 5 37 -) (preferred). DSC on.
and this seems to remain the same whether I have DSC on or off on the R1811. ?
-> 5120x2880@59.02237Hz 175.316kHz 925.67MHz h(48 32 80 +) v(3 5 37 -) (preferred). DSC off.

EDIT:
Quote: @joevt (from Jun 14 2022)
"You have the expected over 900 MHz timing but we can't tell if that's HBR2 with 6bpc (not usually possible on Mac) or 4:2:2 8bpc (unlikely), or HBR3 with 8bpc, or dual link SST .... or HBR2+DSC."
Does that analysis still stand?
/EDIT

Anyway with DSC On, the Blackmagic Test for the SN750 are:
1442MB/s Write, 2800MB/s Read.

If I switch the R1811's settings to DSC Off, I get:
1753 MB/s Write, 2800 MB/s Read.

So I'm presumably using less video bandwidth???

I downloaded a hires grayscale gradient image from Projector Central, and looked at it in detail fullscreen .
The image displays exactly the same whatever the DSC setting.
I can see the 256 gradient steps on an 8 bit gradient, but the 10 bit gradient juxtaposed below it seems stepless.

Anyway I'm happy, whatever I've got as the iMac Pro screen is gorgeous,
and with only two TB 3 ports on an M1 Mac mini there have to be compromises.

I've saved the full terminal pages if I haven't quoted the right info. Thank you.
 
Last edited:
Pardon my ignorance but what is DSC and where/how do you set it? I don't have displays that support it I think so I've probably never seen it.
 
Display Stream Compression. It’s why 5K monitors can display 10 bits per channel on a single DP 1.4 cable.
Before graphic cards began supporting it in about 2019 you needed 2 cables (or embedded streams in TB) as described in (1) above. Each stream carries half the screen data - left half/right half.
Its why Target Display mode was never possible with 5K iMacs

Its only relevant for 5K+ external screens, and Apple totally hides its external display processes.
 
Last edited:
Pardon my ignorance but what is DSC and where/how do you set it? I don't have displays that support it I think so I've probably never seen it.
DSC compresses up to 36 bpp to an average of 12 bpp so a display can have higher resolution or faster refresh rates. 12bpp is the default for macOS. DSC can go as low as 8bpp but I don't think anyone's tried that.

Chroma Sub Sampling 4:2:0 compresses 30 bpp to 15 bpp and 24 bpp to 12 bpp by having color info for only a quarter of the pixels. Only the brightness info has 100% resolution.

DSC is a visually lossless method. It's much smarter than chroma sub sampling. https://en.wikipedia.org/wiki/Display_Stream_Compression

I'm using a DIY 5K 27" display with an iMac Pro screen (new) and a Chinese R1811 v4 video driver board, set to DP 1.4 mode.
So its technically an 8 bit + FRC screen?
Connected by a USB-C>DP 1.4 cable (CableMatters 8K).

The current mode seems to be:
-> 5120x2880@59.937Hz 175.316kHz 925.67MHz h(48 32 80 +) v(3 5 37 -) (preferred). DSC on.
and this seems to remain the same whether I have DSC on or off on the R1811. ?
-> 5120x2880@59.02237Hz 175.316kHz 925.67MHz h(48 32 80 +) v(3 5 37 -) (preferred). DSC off.

EDIT:
Quote: @joevt (from Jun 14 2022)
"You have the expected over 900 MHz timing but we can't tell if that's HBR2 with 6bpc (not usually possible on Mac) or 4:2:2 8bpc (unlikely), or HBR3 with 8bpc, or dual link SST .... or HBR2+DSC."
Does that analysis still stand?
/EDIT
Since you're using a DisplayPort cable that it can't be dual link SST. It's probably not dual link MST because that would be weird (it was only ever a thing when the first 4K60 displays were released). If it were dual link than the script would say (tiled).

I have not looked into how to determine what pixel format an Apple Silicon Mac is outputting since https://forums.macrumors.com/threads/diy-5k-monitor-success.2253100/post-32093817

Anyway with DSC On, the Blackmagic Test for the SN750 are:
1442MB/s Write, 2800MB/s Read.

If I switch the R1811's settings to DSC Off, I get:
1753 MB/s Write, 2800 MB/s Read.

So I'm presumably using less video bandwidth???
Seems that way. You could try other resolutions such as 4K60 or 1440p60 to see how the write speed changes.
5K60: 925.67MHz at 12bpp = 11.1 Gbps.
5K60: 925.67MHz at 8bpc = 22.2 Gbps.
4K60: 533.31MHz at 10bpc = 16.0 Gbps.
etc.

I still think a drop in write speed from 2700 MB/s to 1753 MB/s is greater than it should be if the 5K display is only using 11.1 Gbps. Maybe I'm not calculating bandwidth correctly. Or maybe a different NVMe will have different behavior.

I downloaded a hires grayscale gradient image from Projector Central, and looked at it in detail fullscreen .
The image displays exactly the same whatever the DSC setting.
I can see the 256 gradient steps on an 8 bit gradient, but the 10 bit gradient juxtaposed below it seems stepless.
In macOS System Information.app, it shows the framebuffer depth which can be 30bpp even if the GPU is outputting 24bpp.
Some apps (or the graphics drivers) in macOS can use dithering to make 8bpc look like 10bpc. I don't know how to detect that.
 
I think I'm getting off topic here but Apple native displays in the MacBooks and iMac are 8bpc right? Only the XDR display is 10bpc? Even the iMac Pro from 2017 is 8bpc?
 
Intel iMacs since 2015 (including the Pro) had graphics systems that rendered the frame-buffer at 3x10bit colour depth.
The screen technology is called 8bit + FRC - "Frame Rate Control, which manipulates pixels so they flash two alternating colors so quickly that you perceive any of the shades in a billion color experience."

So Apple's tech specs for the iMac Pro can say "with support for one billion colors."
Only the XDR is true 10 bit.
 
Thanks Paul. I can live with that. I've got 2x 2015 iMac 27s and just picked up a cheap iMac Pro.
 
  • Like
Reactions: PaulD-UK
@joevt I'm noticing a curious situation with my new M3 Max. It's connected to the OWC Thunderbolt 4 dock which is connected via USB-C to a LG 27UL850-W and to an Acasis Thunderbolt 3 enclosure with Samsung 980 Pro. When directly connected to the M3 Max, the Acasis gets better performance than with my old Late 2020 M1 MacBook Pro at roughly 2700 MB/s read and write. The old M1 was only able to manage 2400-2500 MB/s read and write. But things get weird when using the dock. The old M1 managed to get 2200 MB/s write and 2400 MB/s read. This M3 Max gets 1976 MB/s (almost always to the last digit) write and 2700 MB/s read. It can never write any faster.

Is that the expected performance hit one would get from having a single 4K display in this setup? And if so, how come the old M1 performed better in writes when connected to the same setup?

If I leave BlackMagic running and then unplug the display, the writes shoot up to 2700 MB/s. I replug the display and the writes drop back down to 1976 MB/s. Similarly, if I switch the refresh rate to 30Hz, the writes will shoot way up to almost 2700 MB/s. When I switch it back to 60Hz, they go back down to 1976MB/s. Lowering the resolution on the other hand, to something like 1920x1080 (non-retina), does not seem to affect write speeds.

Is this some sort of macOS or MBP firmware bug that I'm encountering?
 
@joevt I'm noticing a curious situation with my new M3 Max. It's connected to the OWC Thunderbolt 4 dock which is connected via USB-C to a LG 27UL850-W and to an Acasis Thunderbolt 3 enclosure with Samsung 980 Pro. When directly connected to the M3 Max, the Acasis gets better performance than with my old Late 2020 M1 MacBook Pro at roughly 2700 MB/s read and write. The old M1 was only able to manage 2400-2500 MB/s read and write. But things get weird when using the dock. The old M1 managed to get 2200 MB/s write and 2400 MB/s read. This M3 Max gets 1976 MB/s (almost always to the last digit) write and 2700 MB/s read. It can never write any faster.

Is that the expected performance hit one would get from having a single 4K display in this setup? And if so, how come the old M1 performed better in writes when connected to the same setup?

If I leave BlackMagic running and then unplug the display, the writes shoot up to 2700 MB/s. I replug the display and the writes drop back down to 1976 MB/s. Similarly, if I switch the refresh rate to 30Hz, the writes will shoot way up to almost 2700 MB/s. When I switch it back to 60Hz, they go back down to 1976MB/s. Lowering the resolution on the other hand, to something like 1920x1080 (non-retina), does not seem to affect write speeds.

Is this some sort of macOS or MBP firmware bug that I'm encountering?

Is the following correct?

MacBook Pro​

Dock​

Display (connected to dock)​

Read​

Write​

M1no25002500
M1OWC TB44K 60Hz24002200
M3 Maxno27002700
M3 MaxOWC TB4no27002700
M3 MaxOWC TB44K 60Hz27001976
M3 MaxOWC TB44K 30Hz27002650
M3 MaxOWC TB41080p 60Hz27002700

It makes sense that connecting a display affects write speed if the display uses enough bandwidth. I don't know how great the affect should be. Bandwidth calculations get complicated if you start looking at the protocol instead of just the amount of data. I would not be surprised by a difference between two different host controllers but it's a little surprising that the newer chip has less performance than the older chip in the case where a display is connected. It could be a regression in chip design or driver. Or it could be a correction to mitigate some bandwidth related issue.

2700 MB/s is 21.6 Gbps (not including protocol overhead).
2200 MB/s is 17.6 Gbps (not including protocol overhead).
1976 MB/s is 15.8 Gbps (not including protocol overhead).

To calculate DisplayPort bandwidth, I just multiply pixel clock by 30bpp. A display that uses DSC would have 12bpp.

4K60 is 16 Gbps. But that leaves 24 Gbps for the SSD. Can the SSD overhead be > 2.4 Gbps? Or maybe the SSD needs to occasionally wait for a DisplayPort packet to finish transmitting.
4K30 is 7.9 Gbps. Maybe the SSD needs to occasionally wait for a DisplayPort packet to finish transmitting so there's a tiny drop.
1080p is 4.2 Gbps. No waiting is needed for this tiny amount of DisplayPort traffic.
 
Is the following correct?

MacBook Pro​

Dock​

Display (connected to dock)​

Read​

Write​

M1no25002500
M1OWC TB44K 60Hz24002200
M3 Maxno27002700
M3 MaxOWC TB4no27002700
M3 MaxOWC TB44K 60Hz27001976
M3 MaxOWC TB44K 30Hz27002650
M3 MaxOWC TB41080p 60Hz27001976

Thanks, that table is almost correct except for the 1080p 60Hz write speed which is the same as 4k 60Hz. So it's almost as if macOS just reserves the full 4k 60Hz bandwidth in that scenario. When I say 1080p I don't mean the retina version of 1080p which is actually 4k meant to look like 1080p; but rather the low resolution version you have to choose from the list of manual resolutions in Sonoma. It's just strange that 30Hz affects bandwidth but the resolution doesn't.

My memory is a bit hazy, so I could be wrong. But I seem to remember in previous versions of macOS on my M1, I could lower the resolution and gain write bandwidth. I distinctly remember having BlackMagic Disk Speed Test running at one point with Settings open when I was using the highest scaled resolution, and it reporting around 1900 MB/s only to have it jump up to 2200 MB/s or so when I dropped the resolution to the 2560x1440p scaled retina resolution. That no longer seems to be the case in Sonoma with this new machine.
 
Thanks, that table is almost correct except for the 1080p 60Hz write speed which is the same as 4k 60Hz. So it's almost as if macOS just reserves the full 4k 60Hz bandwidth in that scenario. When I say 1080p I don't mean the retina version of 1080p which is actually 4k meant to look like 1080p; but rather the low resolution version you have to choose from the list of manual resolutions in Sonoma. It's just strange that 30Hz affects bandwidth but the resolution doesn't.

My memory is a bit hazy, so I could be wrong. But I seem to remember in previous versions of macOS on my M1, I could lower the resolution and gain write bandwidth. I distinctly remember having BlackMagic Disk Speed Test running at one point with Settings open when I was using the highest scaled resolution, and it reporting around 1900 MB/s only to have it jump up to 2200 MB/s or so when I dropped the resolution to the 2560x1440p scaled retina resolution. That no longer seems to be the case in Sonoma with this new machine.
You have to check the pixel clock of the mode. SwitchResX will show the pixel clock of a mode if you double click the mode in the Current Resolutions tab.
There's multiple ways to do 1080p on a 4K display:
- 1080p lo res, not scaled: (30Hz, 68.31MHz) ; (60Hz, 138.56 MHz)
- 1080p lo res, scaled to 4K: (30Hz, 262.81MHz) ; (60Hz, 533.31 MHz)
- 1080p HiDPI, 4K not scaled: (30Hz, 262.81MHz) ; (60Hz, 533.31 MHz)
 
  • Like
Reactions: HDFan
I just wanted to circle back to this and say that I discovered the problem with my slower write speeds. The Thunderbolt cable I had been using for well over 5 years must have been going bad. I replaced it with an Apple Thunderbolt cable and my speeds fully recovered!

EDIT: I thought to do this because my external display wasn't being reliably recognized when connected through the dock. I've seen some people blaming macOS for that sort of issue. It very well might be that your Thunderbolt cable needs replacing. The Apple one is noticeably better quality.
 
The Apple one is noticeably better quality.
There have been quite a few web articles about the risk of using 3rd party USB cables. Here's one discussing an Amazon cable which melted:

https://apple.news/AeR58Rtb_TqCZ7_FAPqdPnQ

Here's a warning about hacking potential (which doesn't apply here):

https://apple.news/AbeYFAzPDSx6wV_mI2d75AA

An excellent video explaining why Apple's USB-C cable is so expensive. In this hard to view shot you see how Apple has added length to a lead in order to make sure that signals get to the right place at the right time.

Screenshot 2023-12-30 at 03.29.28.png

and look how complex the cable actually is:

Screenshot 2023-12-30 at 03.32.27.png

compare it with this one:

Screenshot 2023-12-30 at 03.34.00.png


so you are generally getting a much better cable if you buy from Apple and are wiling to pay the price. Otherwise ..
 
  • Like
Reactions: 43n79w
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.