Well, there seems to be generally three sized phones now (has been like that since HTC and Samsung came out with as many sizes as they have). HTC and Samsung both have the most general sizes (and now thankfully, Apple). The 3 general sizes seem to be 4" to 4.3"....(small) 4.7" to 5.0" (medium)... And the 5.5" to 6.0" ( large). I know...obvious.
. But jumping up only one size from a smaller size makes you go "wow, big difference". Two sizes is another animal all together.
I went from the iPhone 4S, to the HTC One (4.6 in) for around 8 months, then went to the larger (5.9" HTC one max). Wasn't such a size shock, although each step was somewhat of a noticeable size shock that took me a while to get used to.
Anytime you jump a size up, it will be VERY noticeable. If I was coming from the iPhone, I would (after having been there, done that) go for the 4.6 because it will seem huge as well, after using the iPhone. Use it for a year or so, get the feel for the size diff, and THEN feel the 5.5" to see if you like it. The 4.6 will feel shockingly large. I think a lot of people coming from the iPhone directly to the new 5.5 iPhone will be unhappy. It's going to be a HUGE difference, from feel to everyday usage, to just carrying the thing. A lot of things change with such a large phone, just to carry, access and use. A phone that big will force you to change the way you own a phone.
I personally cannot wait for the 5.5" plus(which will actually be a size down for me). I've owned every iPhone since the first, but didn't buy into the 5 or 5s, because I wanted more screen size. Looking forward to coming back to Apple as everything I own is an Apple product, and LOVE their stuff.