While I don't want the 5.5" model to become the flagship, I'm ok with the 5.5" having a slightly faster clock speed because it has a better thermal profile (larger enclosure like the iPad Air vs. Mini Retina). If it has a higher-resolution display due to the larger screen size, then I'm ok with it having a slightly faster graphics chip. What I don't want to see is a dumbed down 4.7" model that is marginally faster than the 5S, or has less RAM than the 5.5" model, or can't play the same high-end games as the 5.5" model or is missing camera features.
Here's the thing: I get that Macs come in different form factors with different speeds, and sometimes iPads do (original Mini was slower). The difference is that, for the most part, there is only one iPhone form factor that fits each user because they have to carry it in their pocket and hold it in a hand. Normal sized people will get the 4.7". Larger people will get the 5.5", and perhaps a few normal sized people who carry it in a bag because OMG phablets. Getting an iPad Mini vs. and iPad Air is about portability on the go. Do you want a smaller, lighter bag with a device that is easier to hold for extended periods, or do you want a bigger screen that is easier to read or draw on? With a phone, for the majority of people (80-90%), it needs to fit in their hand and pocket. There isn't as much choice. When you remove choice, and force people who want to be just as productive, but aren't giants, to use a slower device because it fits in their pants—then you end up pissing off some of your best customers. We're the ones always buying Apple stuff every year. We're the ones recommending to our friends and family to buy Apple stuff. We're the ones that want a high end device.
I'm not saying it's going to happen, but if it does, I'll be quite upset when I buy my iPhone 6 anyway. Actually, I say that half-joking. If the 4.7" is missing a lot of the 5.5" features, then I might just hold off until the iPhone 6S just to spite them. A dumbed down 4.7" model wouldn't be much of an improvement anyway.