Replace the battery then.
It isn't very hard to do it yourself, and you can easily get a kit online with the required tools (00 phillips and a pentalobe driver for the screws on the bottom).
I've done them for $8 battery only, and you should be able to find a kit for around $20 from what I've seen.
Night and day difference. I've still got a 4s that I use as a spare device running iOS 8.3. It will easily go 6 hours of usage and last throughout a normal day (7:00 to 11:00 generally) with a bit left over. Usage for me being iMessage, Facebook, Instagram, decent bit of Camera use, Safari, Notes, Mail, Evernote, Music, Calendar, Weather apps, some YouTube, and the occasional game.
Other than that, you've probably disabled most the battery hogs, screen brightness, BT/WiFi when not in use, background app refresh (minimal), reduce transparency (helps with the lag/jitteriness) and perhaps a few smaller things.
But I'd go for a new battery either DIY, or find a shop. A local one in my city will do battery replacements for $30 on the spot.