I wouldn't be so certain about that. M4 is fantastic, and I am typing this on an M4 Pro MBP, but if you push the machine then M4 runs hotter than M3 simply because M4 has a higher package TDP and is allowed to run at higher peak wattage. In multiple online reviews M4 MacBook Pros turn on the fan(s) sooner and louder to cool off the chip than M3 MacBook Pros.
In a fanless chassis like the MacBook Air, I would expect M4 to get hotter or thermal throttle sooner than M3 version when the system is under high load. If you're just doing email and browsing the web then neither M3 nor M4 are going to overheat.