Some people seem incapable of viewing both sides of a conversation. There are reasons why we are where we are, and there are valid arguments as to why that isn't the end of the story.
I've used nearly every iPhone (or at least one from each year) ever made. I've upgraded annually 17 years in a row and will again this year. I've gone back and forth between small phones and large phones since they started having multiple sizes. Here is what I can offer:
The utility of the phone does not actually increase as the display size increases. It is still the same device, performing the same tasks, and (most importantly) being used the same way. Utility does not increase until you start to get somewhere up around 10 inches, tablet size. At the point it has gotten big enough to change the way you use the device, which is why there are distinct size ranges for phones and tablets. Size however is an important factor to the limitations of physical technology. The larger phones are not really about the larger screens. They are about the larger battery, and to a lesser degree, the better cameras. As we go forward into high density batteries, it may be possible for Apple to make smaller iPhones that have acceptable battery life. But physics will always be a thing. No matter how good a small phone can be, a larger phone can be better, because there is more space for the better. Whether that is battery, camera, you name it. More space = more better.
This, along with price, have been the main driver for Apple. They can make a better product if it's bigger, and they can also charge more for it. This is unavoidable. Bigger almost always means more expensive for Apple products, and they couldn't resist pushing into higher priced iPhone territory.
That said, the upcoming iPhone Air gives a glimpse into the fact that Apple doesn't necessarily enjoy how bulky and heavy the larger iPhones have become in order to accommodate the largest displays, batteries, and cameras. They desire to reboot this somewhat, and start bringing iPhone design back to the thinnest and lightest possible version of itself. They are opposed to it getting too large.
I think the design goals of the iPhone Air tells us that this is the smallest iPhone (in terms of both height, depth, width, and weight) that they are comfortable with making at this point in time, to still deliver the minimum standard of quality they're looking for. Making a fat brick just to bring the screen size back down below 6 inches isn't in their heads at this point as an acceptable product, and I agree. But I think they will once they feel that they can.