Well, even if you don't, it'll still be open. In OS X, Applications can be open without having any windows. In addition to browsing through files, Finder handles how your desktop icons appear, among other things.
If you quit absolutely every application in OS X, including Finder (which is posisble), Finder automatically opens anyway.