Most likely the 2.4GHz band is saturated, which is where you get your speed drops.
Except it’s not, and my other devices work fine.
I also use a Ruckus R500 AP which performs background scanning to pick the best available channel. Like I said, I have 3 other Apple TV 4’s s that will pull 200-275mbps while the ATV 4K pulls a fraction of that.