Even Windows programs aren't completely gone when you uninstall them. This is no different on the Mac. However, I would say it is much easier to use AppZapper to remove applications than an uninstaller program in Windows.
Hmm.. I've made it this far without an uninstaller on OS X, why can't you, OP?
I think it's more for a sense of tidiness then actual performance. 10 8kb leftover files scattered here and there (typically out of sight) doesn't negatively impact you at all.
With the exception of app that drop kernel extensions, or system startup items, there's no real need of an uninstaller.
I've leanrt over the years that Apps that you drag and drop- i.e. to the desktop or Applications, offer the cleanest way to getting rid of them- via drag and trash.
Apps that requiring you to run an installer tend to be a little messier.
Granted some Apps do require you to run an installer as the files need to reside in specific folders to work properly- i.e. iTunes Visualizer Plugins, KeyNote Themes, Screensavers etc. But often the installer is just there for your conveneince, but you can also locate the folder location manually.