I went from 5 to 5s and regret it immensely. And thats why I returned the ever elusive Gold 5s that I hunted 2weeks for and went back to the 5. So here's my reasons for going back:
1) Forward Thinking - That definitely applies to the 5s because things like TouchID, 64bit and the A7/M7 make this a future proof'd phone. But thats also the exact problem. If these features won't be properly supported for another year or so then why not buy next years phone?
2) Camera - This is my biggest reason to buy so it's also my biggest disappointment. Yes, the camera is better than the previous version... when taking pictures of mountains. But in the real world we photograph people in poor conditions. The lack of actual control over shutter speed or ISO means we have the same blurry and grainy photos that we've always gotten so the 5s didnt really blow the 5 out of the water. (BTW. I know these are software complaints but the inability to export Slo-Mo into iMovie.app is ridiculous and forcing "Burst Mode as default" has really hurt the iOS7 camera experience.)
3) Touch ID - While I know several people who have professional needs for an always locked phone, for us non-spies then most people lived just fine with a 4hr passcode. So, as much as I love the tech, it's hard not to label TouchID as more than a gimmick. Where I do see this changing is purchases but again, we've got a while before that gets implemented.
4) Screen Size - Technology evolves and so do the tastes of users. Larger screens are inevitable because in the real world then a bigger screen is more visible, usable and doesnt make a phone less pocketable. There's no doubt Apple will do it right so I want my contract ready for a bigger-screen iPhone 6.
5) iOS 7 - The software upgrade from iOS6 -> iOS7 felt MUCH more significant and satisfying than the hardware upgrade iPhone 5 -> 5s.
So, thats my reasons. If going from a 4S I'd definitely say the 5s is a worthy upgrade. But for 5 owners then it's definitely a tough decision because you're paying for pretty much the same product you already own.