Wow you just gave me the last piece to my puzzle, thanks for that! I've had a similar setup to yours that you described in your earlier comments but had no audio/camera and just video. I flipped the Cable Matters USB-C cable and it actually worked.
It freaked my XDR out though and the fans started running at full blast. Exhaust air was cool and the XDR was still working fine so I'm assuming it wasn't overheating. Fans came down to normal with a quick reboot of the monitor. Other than that, I've had no crashes and have been using the Studio Display XDR with my 5090 PC, M3 Ultra Mac studio and PS5 Pro.
My Setup (Using the 3-port version of the Cable Matters switch
https://www.amazon.com/dp/B0DLJT49Z3):
Mac Studio Thunderbolt —-> StarTech Thunderbolt 5 Dock -> DisplayPort/USB-A to USB-C Cable —-> Cable Matters USB-C Switch
Windows PC —-> DisplayPort/USB-A to USB-C Cable —-> Cable Matters USB-C Switch
PS5 Pro —-> Club 3D CAC-1335 > DisplayPort to USB-C Cable —-> Cable Matters USB-C Switch
(Can only get 1440p120hz with this setup no HDR at 120hz)