I went from a 3GS to a 4S, so of course I love it (retina, form-factor, speediness, camera). When my wife uses my 4S compared to her 4, she says she can tell that the camera takes better pictures (and she's not alluding to the mega-pixels). But, I don't think she notices any other differences. I actually find the 1080p video annoying because it means all my video clips are huge even though I don't really need such a high res; it really chokes AirPlay, unlike iPhone 4 videos.
I can tell that my 4S loads apps, websites, etc. a lot faster than my wife's 4. And I too can tell that it takes better pictures. I feel better about the antenna, but that's really all just based on the advertising propaganda. The simple reality is that I havent had call trouble, but neither has my wifes 4 (though she does use a case while I dont).
Siri I find to be nothing more than a fancy gimmick. Requiring internet access for any and every command, even simple, local-to-phone tasks, is a real handicap to its usefulness. And as much as the advertising would like you to believe it just understands you, the reality is that it's no where near as intelligent as it needs to be. It's not intelligent at all, really; it doesn't learn. It doesn't really piece together grammar. It feels like it simply has a huge list of programmed phrases, and huge as it is, half the times I ask it something, it just doesn't get it. And I'm not trying to stump it; I'm just trying to use it the way it claims it's useful for.
Overall, I don't think the 4S is a compelling upgrade from a 4, at least not unless you could upgrade for a minimal cost by selling your 4 for a good price. Or unless you highly value how quickly things load. I never really felt like my 3GS was slow, not even on iOS5, not slow enough to annoy me at any rate, but I can still appreciate just how much faster my 4S is.