I just got a Nexus 4 and honestly, I miss my iPhone 4. Even though the N4 is much faster, I always get the feeling that Android always needs more refinement to the user experience. Also, the quality of apps...although it has gotten a lot better from the last time I had an android phone. iOS may be a bit stale now, but its solid as a rock and everything is refined.
For example, when I first started playing around with the N4, I couldn't believe that Jellybean didn't have lockscreen notifications (I hate lockscreen widgets, just tell me how many unread texts and emails I have. Nor could you have a different lockscreen wallpaper. I actually had to spend a total of $5 on apps to make this happen (Widget Locker and Missed It!).
Another example would be the email client. I started using the Gmail app and while it can render the email properly to fit to screen, pictures don't automatically download. There's no setting for this either. I can only enable automatic picture download on a sender by sender basis. A little annoying, but is this really neccessary?! I tried other apps, and while some are able to display pics automatically, they can't render the email to fit to screen. Who wants to scroll sideways when reading an email?!
These are just 2 of the annoyances that I have come across that may very well make me switch back to the iPhone 4.