The iPhone 4s went from single core to dual core CPU, and can run the latest OS 8 and 9 while the iPhone 4 is stuck on iOS 7. It also had a very tangible camera update from 5 to 8 mega pixel.
The 5s got TouchID as well as a 64 bit CPU that is much faster with 64 bit code, that will likely let it last into later iOS versions than the iPhone 5.
Big feature that people can see/use for 4s was Siri, but just as important of a hardware change was going to a dual-core CPU, which allowed for better and newer features and further progress down the line.
I really try and stick with the S models since these models get the hardware bumps while the number variants are just a change in form factor with minor hardware changes.