Ugh I had this exact same issue this morning! I usually wake up between 7 and 8, but when I was rousing I noticed it was a bit bright. I tap my iPhone to wake it to see the time and it's 5:50am, and the brightness was the screen of my new iMac across the room. I grumbled out of bed to put it to sleep again.
At first I thought tapping the phone woke it up somehow but the iMac must have already been awake or else the brightness wouldn't have woken me up so early. I experienced this same issue on my 2017 iMac for awhile and I made some adjustments to settings such as what
@AZREOSpecialist pointed out and it stopped finally last year. When I upgraded to the 2020 I set it up the same way, with one exception.
Do you also have an external hard drive that you use for Time Machine backups and your TM setting is set to Automatic? This was what completely corrected the problem for me in my case on the 2017. On the new 2020 when I set it up I thought maybe Apple fixed that weird waking up issue I was experiencing, so when I set up TM I used Automatic again. Seemed fine for a few days. Then a few days ago I get home from work and the iMac is on again after I put it to sleep when I left in the morning. Sleep is set for one hour so it must have happened just before I got in. And now this morning it woke me up when it woke up! So I disabled the Automatic setting and I'm hoping it completely stops.