Yes I’m having this issue as well. I wonder if the latest update is the culprit. It worked fine yesterday so not sure what happened. I just turned off reminders in iCloud on my phone and turned it back on and the watch is currently slowly getting the data back but wow it’s so slow.
On another related note, why is the initial iCloud sync so slow? This to me is really unacceptable. Google drive in my occasional use initially syncs incredibly fast so it’s annoying and weird that iCloud takes so long. If Apple wants people using their default apps and paying for the privilege to cloud store their documents then they should really fix this.