Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
I'm assuming you are referencing Finder windows and not App windows. I see the same behavior with my Mojave system. High Sierra and earlier did not reopen the Finder window.

I'm talking about apps like Spotify, Messages, Chrome, etc.
 
I'm talking about apps like Spotify, Messages, Chrome, etc.

The Chrome default installation is to automatically start up when MacOS starts up. I think I disabled the app from starting up under System Preferences > Users & Groups > (select the user account) > Login Items.
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.