I had never put my 27" to sleep, but mine started doing this, too. Figured out it was Bluetooth pulling it out of sleep. Go into Bluetooth options, and then "Advanced," and make sure the checkbox is unchecked for allowing BT devices to wake the computer.
OP is using a wireless (bluetooth) keyboard and mouse. Disabling this would not allow the keyboard or mouse to wake the computer (at least this is how it works on my MBP).
Disabling the "Wake for Network Access" option in Energy Saver Preferences (System Preferences) should fix this problem.
If not, try a PRAM and SMC reset.
Disabling the "Wake for Network Access" option in Energy Saver Preferences (System Preferences) should fix this problem.
If not, try a PRAM and SMC reset.

🙁 I like this option as I have multiple computers I can access the iMac without manually waking