Issues with sleep on M2 Max Macbook Pro

[kasakka]

I run my M2 Max Macbook Pro 16" with lid closed, connected to a keyboard, mouse, audio interface and two Samsung G70A 4K 144 Hz external displays using MBP HDMI port and a CableMatters USB-C to DP 1.4 adapter.

I'm having two issues:

1. It's hard to get the computer to sleep from the Sleep option in the Apple icon menu. Unless I press Sleep and immediately take my hands off my desk, it seems like even the slightest bump of the mouse from just having my hands on my desk will wake the computer back up right away. It seems to sleep if I just let the sleep timer run.
2. The computer seems to keep waking from sleep randomly even though e.g "Wake for network access" is set to "Never." I can see this from my Samsung G70A monitor connected to the HDMI port waking up randomly even when I am not in the room. It wakes up to a blank display and doesn't show e.g lock screen. Then after a few seconds the display goes off again.

The odd thing is that I had a 2019 Intel MBP 16" with the same setup and it never did this. It only started occurring when I swapped it for the M2 Max MBP 16". If I swap my ITX desktop PC (13600K+4090, Win 11) to this setup, the issue does not happen.

Does anyone know what might be the cause of this?