When I had the 4 and 4S I thought the screen size was perfect. And right before the 4 I had a 4.3" Android (which I also thought was perfect in size).
When I switched the 5S I thought that too was perfect. Holding a 4/4S after that seemed very small (short and fat, more like it).
Then yesterday at work they gave me a new company phone - Nokia Icon, which is 5" - and I LOVE it!
But the thing is, the shell of the phone isn't really all that big! That's b/c the screen is nearly edge-to-edge. AND the top and bottom parts are narrow as well. So the majority of the face is the screen.
Looking at my 5S and seeing that thick top and bottom now is very...meh...

I thought I would be happy w/ the 4" 5S as THE perfect size but after handling the 5" I hope the 6/6S/6C (whatever) is AT LEAST 4.5" big.
The only other thing to keep in mind is these >4" phones are NOT 16:9, otherwise they'd be way too tall...
As for one-handed operation on the Nokia, the only part of the screen I can't reach w/ my thumb is the top-left corner. So if this were an iPhone screen, I wouldn't be able to tap that icon or pull down the notifications unless I adjusted my grip.