I actually went the other way on the iPhone - SE for me.
I had a 6s+ prior, and while I loved the device it was so unwieldy to carry around. I feel like my usage didn't change much going to the SE, but at the same time I also think I focus more on using the right device for the task at hand.
My setup has moved from big and rather unorganized with lots of overlap to more specified with minimal overlap.
Former setup:
27" iMac, 11" MacBook Air, iPad Air 2, iPhone 6s+, iPod Touch
Current setup:
15" MacBook Pro, iPad Pro (12.9"), iPhone SE
For me it gives three set categories, and of them the iPad Pro admittedly gets the most use, as I find it the best option for casual browsing and the like. It's also the most comfortable on the couch!
I may check out the larger iPhone 7 when it drops, but overall I'm really happy with the SE. Between the battery life, quick charge time, and barely there form factor it's a great fit for me.