Coming from Windows, you're going to have a hard time wrapping your head around those 3 little lights. 😀
First get used to Applications that stay running with no windows. And yes, it's quite common to have an inactive window from another application show directly under the menu bar of an application you thought you closed.
I installed a 3rd party app called RightZoom. It forces the window to consume the entire display when pressing the Green "optimize" button (instead of merely resizing the window to fit the text).