I haven't downsized, but my wife has a 6 so I do have some opinions.
When I first compared the 6 and the 6 plus, the plus was the "no brainer" option for me. Part of that was because I didn't feel the 6 was really a large enough size increase.
After having used my wife's phone, the 6 is definitely more usable with one hand; I am not generally a one handed user, but I have tried the 6 since so many people make such a big deal about it and I can do everything one handed on that device, even without reach ability.
Battery life is improved from here 5S. I know this because there were days when she was calling me from work letting me know her phone died (she works late and about a 30-40 minute drive so I will sometimes check in on her if she is later than normal, or to see if she is on her way yet with find my iphone). Her 6 has yet to die and is always above 20% at the end of the day.
I will say that the screen is just "nicer" on the plus. I am not going to go into specs or anything, it just looks nicer on virtually all content. She even commented on that one day playing with my phone, but said the plus would just be too larger for her everyday use, so the sacrifice is worth it.
As far as performance, I find her device to run things smoother. Not everything, but there are some quirks with the 6 plus that I have seen that I can replicate very easily (lag, frame rate drops, whatever) that I haven;t come across on the 6 at all. That said, I have not used the 6 nearly as much as the plus, but she states she hasn't noticed issues; I fully admit that different people are going to have different sensitivities to these sorts of things.
Overall, if I would trade in my plus for a 6 I don't think I would be overly heartbroken, but I would miss it. My usage tend to be very little single handed use (typing). I read a lot of news, stream music, browse the web, watch a movie here and there in bed. I think all of those things are just better on the plus because a bigger screen just makes those activities more enjoyable. As mentioned though, the 6 wins the one handed use argument hands down in virtually every scenario for me.