I personally don't jailbreak. I am tempted sometimes, but the key reason I don't is because I used to be a Windows Mobile user (Motorola Q), and I learned the hard way that the more you customize the phone with themes and such, the poorer it performs. When I first got the Q, I went crazy customizing it in all sorts of ways. Several months later, I found myself stripping nearly everything off that ran in the background, so I could get decent performance from the phone again. That says something.
The cold hard reality of all smartphones is that none of them manage memory or multitask real well. It's an inherent limitation brought about by the need to manage battery life. The processors and the memory bus can only be so fast, and for cost reasons you can only have so much application RAM on board. EVERY phone has this limitation, the iPhone is no exception.
Features are important to me, but the phone must first and foremost be responsive. Even out of the box, the iPhone is not always as responsive as I'd like (though mostly it's fine). I'm not going to make it worse.