Perhaps your friends have lots of Apps running in the background? My Froyo'd 2.2 Nexus One is noticeably faster than my iPhone 4. This includes navigation of the menus, changing homescreens, browsing, app launching, etc.
A perfect example, I use Foursquare. I have the Foursquare app on both my iPhone 4 and my Nexus One.
If I'm connected to Wifi and I have the two side by side, I will launch the app on both devices at the same time. The iPhone 4 takes about 7 seconds upon startup to launch the app, and then locate me.
On the flip side, the same process takes about 2 seconds on the Nexus One. (I am seriously not joking) While this specific example might depend on GPS chip, etc, other apps such as Pandora and Twitter exhibit the same behavior on both platforms, and the Nexus One always has the upper hand in terms of performance and load times.
iOS is a great experience, however Apple put a lot of those cool menu animations in to make it appear the device is loading faster. The fact of the matter is if they stripped out the menu transitions, and you just hit the button, it would still take a second or so to change menus. (often times much longer on the iPhone 3G and below) The animations are put in place to mask this behavior. In fact, this is very obvious sometimes. When your phone is lagging and you hit a button to open up Messages or Settings, sometimes you will quickly see a blank background for a second or two. (the blue/gray color) This behavior is exhibited because the phone is still trying to process/render the next screen.
On the Nexus One? You hit the menu item, and it's instantly there. Literally instantly, faster than you can blnk. No lag, nothing. Oh, and if you don't like that split-second menu transition that Android DOES have by default, they allow you to disable it in settings. (and subsequently, the load time improves)
I'm still debating on returning my iPhone 4 or not, primarily based on principle due to Apple's recent PR response. They have a clear antenna defect, but they can't admit it and I don't like that. The phone itself is great, and it does get better battery life than my Nexus. However the two big missing items for me are Google Navigation and free Wifi-Hotspot. (Froyo - and I'm 99% sure the iPhone will never get either of them)