No, I mean by right clicking in the Window. But, CMD+J does the same thing. It should be holding the settings from there.
I was going to suggest trashing the Finder plist file, which still might be a viable choice. But, when I looked for mine, I had 25 of them! Of those, 24 had an appended six-character suffix of random letters and numbers. I moved them all into an isolated folder outside of ~/Library/Preferences to see if Finder's behavior would be affected.
How often do you Restart? (Is that what you mean by "log back in"?)