I'm not sure what "memory diag" is - please share the exact command or where it's located. If you are using an app on the App Store, I'd encourage you to drop that and focus on Activity Monitor; that's built-in, free, and standard for easy comparisons. And it's well documented, which helps, as 'cool memory utility app' developers can sometimes take extreme liberty at memory functions for underhanded reasons.
Your comments are so general on DOS, OS/2, etc. that I'm not sure what you're trying to say. DOS memory management was junk.
32 bit Windows NT did an outstanding job managing memory, with full protected memory (and preemptive multitasking) long before that came to MacOS. The memory technologies in it are exactly the same (granted, 25 years older) as what's in MacOS (and Linux, Windows 11) today.
I'm not sure who would agree that Intel Macs did a better job of memory management than ARM Macs. Most of the arguments I see about 8GB Macs suggests that memory management has gotten better, not worse.
I'm not sure what you're saying about poor MacOS memory management. What's the exact problem that you see, please? The point of an OS is to manage memory allocation, giving it to apps as they demand it. And in some ways, that IS a free-for-all, driven by you, the owner, as you choose what apps to run.
Again, please define "ceiling". Open Activity Monitor, flip to the memory tab, and look at memory pressure. Is there any, or is it just green?