I fixed it by doing the remove/reset procedure. When It came up it was already on 16.4. I did notice that when I tapped on the Update button, I could very briefly see "Installing" then it went back to Update - so the download completed but it got in some sort of state where it was unable to install the download. So this was not one of the scenarios where the download doesn't complete. Anyway, fixed.
Wow, I will also say, searching for solutions on Google, this happens A LOT and has for years - probably something Apple should address to make things easier for people using HomePods!