As someone who used a 6+ for two years who purchased the 4.7 this time...
it
is all about the size, and there are pros and cons to both. If the camera is what you're worried about, don't - the camera on the 4.7 is extremely good.
The 4.7 is just flat out easier to hold. I find that I like the 4.7 more when laying in bed before I go to sleep, while driving (I mean, uh, I don't use my phone while driving

), putting it in my pocket, how much ROOM it takes in my pocket, etc. Basically, it's better anytime I need to use it one-handed, which it turns out is a lot. I know this because of how much easier using my phone has been in the past 3 days.
I mostly miss the 5.5 for taking pictures, looking at pictures, and watching short videos (I'm not one of those who really uses their phone for longer-term watching of anything). The screen is great in that respect, but for ME, the 4.7 makes sense. It might make sense for you too, and it might not. Added screen real estate is definitely nice but I like handling something smaller more.