I think that's definitely the way to go. Apple could 100% make the next HomePod Mini support Apple Intelligence natively but in my opinion it would be a pretty crazy thing to do because people seriously bought into smart speakers probably have one in most rooms in there home (like me) and personally to cost-optimise my setup I would not want every one of my HPM devices to be higher cost because of the extra processing power that Apple needed to put in to run Apple Intelligence. I'd far rather have your (and my) concept of putting computing power capable of running Apple Intelligence in a single device on my LAN.
Where you and I might differ though is that I lump the full sized HomePod in with the HomePod Mini in that I might have a few in rooms where I want the best HomePod sound I can get, in some rooms even stereo pairs, so I wouldn't want my full-sized HomePod purchase price increased due to more processing power. I do accept though that some smart home users might not want Apple TV. Maybe a solution would be two versions of the full-sized HomePod, one able to run Apple Intelligence locally and the other not?
Maybe that's too messy but - purely my personal preference - one way or another I would prefer the option to get even the full-sized HomePod only provisioned with the compute power appropriate to an edge device (thus keeping the retail price down) and have it hand off to a central AI compute server (e.g. Apple TV) to handle more complex stuff. Easy for me to say I suppose because I plan to buy the new Apple TV as soon as it is available.