Computers are complex systems, and the OS is just one factor contributing to system performance. As a computer ages it accumulates apps, kernel extensions, browser plugins... and some of that becomes obsolete when you update the OS.
So yes, "freezing" your system on an older OS means that those older apps, kexts, and plugins won't be left in the dust. But the flip side is also true - newer apps may not be fully debugged for older versions of the OS.
Meantime, even if you leave your system exactly as it came from the factory - no new apps, no new nothing, system performance can degrade due to accumulated disk errors, caches that didn't flush when they were supposed to, etc.
"Change nothing" or "stay on the old one for as long as possible" may seem to be the lower risk course, "I changed something once, and it turned out poorly, so I'll never change again," but on the other hand, "No risk, no reward." And for many, they'll face a steep learning curve when they are finally forced to assimilate four years or more of cumulative updates.
Back in the early days of MS DOS (when I was IT guy for a large office), I came to the conclusion that every $50 update was worth the cost - there were always new features that ended up saving time and money. And I still find that true today. (More so, because Apple's not charging for updates.) Example, Mavericks brought RAM Compression to OS X - that brought an immediate boost to many older, RAM-starved systems. The expansion of iCloud/iOS integration with OS X (iCloud Keychain and SMS forwarding, WiFi cellular calls, for example) have been great time-saving conveniences (no need to switch back to iPhone - and the iPhone keyboard - to reply to SMS messages, answer/place phone calls, etc).
I have an early 2008 iMac running El Capitan quite nicely. I've had to do some work from time to time (trashing caches, searching out obsolete startup items, running disk utility, etc.). I recently did a full erase/reinstall/restore from Time Machine. Currently, it's running far better than it ran two years ago, when I replaced it as my primary machine. At that point, it was running so badly, anything but web browsing was painful.
So... Was Mavericks a horrible OS? Is El Capitan brilliant? If one ignored all the other factors, one might come to that conclusion. But you can't ignore all those other factors. Even Macs need maintenance.