Yes!
Exactly what is happening to me. Today my watch was 100% at 7am and now at 9pm it is at 71%
Other days I can be 100% at 7am and dead by 1pm
I've been trying to troubleshoot. My comments so far:
1. It's not 10.4, happened to me on the version prior. However it had not happened on any version prior to 10.3.1 I believe.
2. It went away for a while (a week) on update to 10.4, and I thought it fixed, which may be why people say "restore solves" but seems unlikely, as it returned.
3. Sometimes the watch doesn't charge immediately when I put it on a charger. Fiddling with the charger or restarting the watch solves this. Related?
4. I haven't figured out any combination of activity/reason. I do not use any apps, and while I use music / workouts, those both don't seem to matter. On days the battery burning, it loses 1% every 5 minutes when sitting still (or something absurd like that), and on days it's ok the battery is remarkably good, regardless of my activity level/use.
Have you found any triggers?