It all depends on what you're doing and how you use the phone.
If you do nothing but browse basic websites, check mail, and text, there won't be much of a difference. If you play intensive games or use apps that require a lot of power, it's noticeable.
As more apps are converted to 64 bit, the speed difference will become more apparent.
It's faster than my 5 for sure, but it's not blowing my socks off. I certainly wouldn't buy it solely for the speed. The fingerprint sensor and the camera are worth it if you can sell your 5 to cover the cost of a $199,$299,$399 upgrade. It would definitely not be worth spending full price to go to the 5S from the 5, unless you have money to burn, even then...
My 5s seems to have a yellowish tint or warmer color compared to the 5. At first I didn't notice it all that much till last night when it was darker and I was doing side by side comparisons when setting the phone up for someone else and checking the brightness settings. I'm sure I will get over it, but there did seem to be a noticeable difference between the two.
I might upgrade. I know I know it's useless but hear me out, I could just rejoin at&t under my moms plan and get the phone for $200, which isn't a problem because I have $1500 just waiting to be spent in the refund my college gave me. I spent the other $1105 on a new shiny macbook pro and magic mouse lol.
Selling my phone at radioshack or Target covers the $240/250 ETF fee on Verizon so there is really no cost at all.
I notice the difference with Infinity Blade 3 between my iPhone 5s and iPad 4. But normal stuff like web text and video I don't notice any bump between my 5 and my 5s