A desktop machine with an ultrafast SOC like the M4 without active cooling is essentially that, crippled.
You never hear the fan on a M1 mini unless you’re really pushing it, so I don’t see any real benefit of removing active cooling. It makes sense on an ultraportable like the MBA, but it serves zero purpose to release fanless desktop machine, other than making some silly marketing spiel. And of course, upselling your “pro” model with active cooling.
A fanless, port starved base M4 mini would be a huge downgrade than the current base M2 mini, no matter how Apple tries to spin it.