My Android history
2.3.6, Gingerbread
4.0.4, Ice Cream Sandwich
4.1.2, Jelly Bean
My experience with updating my firmware hasn't always been sweet. I'm conservative even updating my apps.
Nokia N82 - Symbian S60. Updated the firmware and the excellent camera with Xenon flash it once was got worse. Purple haze or overprocessed photos. Games got laggy. Batt life also got worse and indicator will tell me full charge to low batt suddenly without any warning.
iPhone (first gen) - Once iOS2 hit, phone was never as quick or smooth as before. Once iOS3 hit, using it was almost unbearable. Two generations later is when you really start to notice.
iPhone 4 - Virgin iOS4 was the smoothest and fastest. I was forced updating to iOS6 after three years on iOS4 because a sim unlock was required to restore and update. Then it got a bit laggy with animations and there was a bug when sometimes the screen won't turn off unless you tap the power button for it to sleep. I also miss that iPod app and hated bloat like Newsstand. But it was needed just to get the unlock and install newer apps I wanted to use. But it was never as fast and smooth as when it was on iOS4.
Even iTunes got worse to me and for PC Windows users after iTunes 9 with the 100% CPU usage.
I prefer sticking to the original software/firmware it came with as that was the first it was originally built on and generally the lightest, fastest, and fairly stable. Updates offers bug fixes but sometimes adds new bugs. I'm on Android 4.1.2 and very much like it. Stable and smooth. I sampled 4.2.2 on my girl's Lenovo, and I don't really care for JB 4.2's features like Widget Lockscreen and Daydream. It lost some of the butter. Galaxy Nexus and Nexus 7 (2012) owners had a nightmare experience going from 4.1 to 4.2 but most low-end phones with Android have 4.2.2 out of the box. Jelly Bean 4.3 wasn't distributed enough and was skippable since KitKat was avail shortly after.
Currently, Jelly Bean 4.1 is the most popular/distributed version of Android. I'm hoping KitKat 4.4.x is just as good as I find updating software becomes worse. We try to get these newer features that we may or may not want but also get lag, glitches, worse batt, bloat, app incompatibility, etc. What I do miss from the pre-4.1 days is the ability to wake the screen up using the volume button using the No Lock app and moving apps to SD card. ICS was the last to do that.
I will eventually update because I want to try out AcDisplay and KK has a lighter footprint than JB. Jelly Bean needs 1 GB of RAM as a bare minimum and ate up more RAM than ICS. But perhaps I can also get a Nexus 5 which originally came with KitKat or OnePlus One which has CM which I can apply most themes I like. Carriers also mess with the updates. I found Android the most mature and stable once 4.0/4.1 were released. KitKat could be the best update since Jelly Bean 4.1 or perhaps a bad one. Either update for new features and apps or stick to the software that is older but reliable and proven.