It depends on what you have installed. Personally, I only have BiteSMS and SBSettings. My phone runs perfectly.
However, if you are theming the hell out of the thing and are running 148 tweaks at once, you can expect a downgrade in performance and stability.
A gun isn't necessarily dangerous; it's about how you use it.
*looks at ip4 that's themed the hell out of and loaded with 138 packages (currently) with no performance or stability issues as compared to husband's stock ip4*
Well, those last ten tweaks must be the difference then! I better never load anything else!
Seriously, though, IF you're diligent about researching before you load something new, if you have the patience to wait while others guinea pig the latest and greatest for you (including actual JB upgrades when they're released), if you're careful about only loading one new tweak at a time and only updating one at a time, and if you pay for your apps so you're not dealing with inherently broken software, it really is a non issue. Note the common theme among all of those is exactly what labman suggested: PATIENCE. (Oh, and of course, save your blobs and back up your critical data regularly so a restore isn't the end of the world!)
The only really bad experience in over a year was a tweak that I *had* researched and thought I could get away with--turns out I should have listened to my elders here and never installed it. And because I'd done enough research over time on how to handle such things, I was able to escape the issue and delete the tweak without having to restore--unlike newbies who panic at the first problem and then make the situation worse. Other than that, the few small problems that have cropped up have been easily handled (or have been due to things outside the jailbreak altogether). My way over the top tricked out phone shows no degradation in battery life or performance when I do identical tasks side by side versus my husband's totally stock phone on 4.3.5, so it's a little hard for me to buy into the whole "jailbreaking will kill your phone's battery/performance" argument, at least on an iPhone 4.
And my obnoxiously overpersonalized iPhone 4 also makes a way more useful tool than my husband's stock iPhone 4. Even he admits he covets the functionality I've added to it--he's just too timid to JB it because I nightly fill his head with horror stories from the people on this board. LOL (I saved his blobs in case he ever changes his mind...)