Some of us here on forums and such perhaps, but we are certainly not representative of 95% of the android consumers out there who just use their phones as they come out of the box.
For those folks - they should have an expectation that a flagship phone should perform without any modifications.
If a manufacturer has added features they should work out of the box and not at the detriment of the users experience by hobbling its performance.
As Robster said it is why many folks coming from iOS to Android may find themselves feeling a little cheated by the performance on their device given it's 'massive' spec sheet difference to their iOS device. There's no point having all the specs if your making them buckle under the weight of your OS.
I really dig my S5, but performance wise it is slower than my S4 GPE in general use - and for a new gen phone, it should feel faster not slower. That's all down to the OS, and like I say it's a shame because Touchwiz launcher part is in itself grand, its the multitude of crap under it that no matter how much tinkering even those of us in the know do, still hold it back from 'excelling'.
It's frustrating because you know the 801 is a screamer of a chipset, and in the S5 it doesn't scream, it just hums.