A TB4 dock is just a USB hub except when you connect downstream Thunderbolt 3 devices. There's no way around that.
The only peripheral TB4 chip is the Goshen Ridge. I think the USB layout is always: "one USB hub with 10 Gbps USB upstream and four 10 Gbps USB downstream ports" even if you are not using USB tunnelling (USB tunnelling happens on M1 Macs when the TB4 dock is connected first - you can disable USB tunnelling by connecting the TB4 dock downstream from a TB3 dock).
USB tunnelling means the USB controller of the M1 Mac is used to communicate with the USB hub in the TB4 dock.
Without USB tunnelling, PCIe tunnelling is used to communicate with the USB controller of the Goshen Ridge.
Now, I suppose in the PCIe tunnelling case, Intel could have made it so that the four downstream ports were not part of a USB hub - instead they could have been part of the Goshen Ridge USB controller itself - so then they wouldn't be limited to 10 Gbps USB total - they would be limited by the tunnelled PCIe - ≈22 Gbps. but I guess that would have added more complexity.
Maybe someone will make a superior USB4 controller chip.