I recently purchased a MAC Mini P4 Pro with the 10 gig upgrade. The Port is total trash. Even when I can keep it connected, performance is all over the place. Upload speed is below 5mbps when they should be at 41.5 mbps for my internet service. Down loan sometime runs near full speed at 1.2 gbps (it should be 1.4 gbs) but often drops to under 100 mbps. I have a $24 USB c 2.5 gig adapter that runs perfectly all the time. I have tried many different cables (some very expensive) and both 2.6gb and 1gb switches but get the same results. the eternal Wi-Fi often take primary position when the port is acting up. If I disable the Wi-Fi to keep the ethernet port the primary then I lose network access all together even though the port shows connected.
Is internet speed the only way you are measuring port performance? Often it doesn't matter what your carrier quotes you as their top speed, at least with my carrier, they are never at peak (1gps), and I am happy if I get 800 mbps, often its far lower. Depends on how much activity is on the internet, not just my house.
As has been stated, the best way to get consistent fast performance that reflects on the port is direct connect hardware devices like a NAS or DAS, and even then it and your cables and your switches (which sounds like you have covered) must be appropriate.