After reading the original headline, I realized I wasn't sure what version of iCloud I was running, and it wasn't immediately obvious how to find out. It turned out I had to look in "Apple Software Update", cancel before the "checking for updates" dialogbox finished (because it would auto-exit otherwise), and hunt around in the pull down menus for a while before I could actually find out.
For me, it turned out I was running 7.21.0 (from 11/16/2020).
In order to upgrade, I had to log out of iCloud on windows, use add-remove to "modify" the installation (and then uninstall from in there), before using the Microsoft Store to install iCloud for Windows.
This process also removes the "Apple Software Update" software, so be careful if you also have iTunes installed (I guess I had already used a similar process to upgrade iTunes using the Microsoft Store).
So, while this is the end users' problem, it is pretty unfriendly that iCloud just stopped updating at some point. I kind of wish it had informed me that "Apple Software Update" was out of date.