Huh. WB was one of the very first things I installed when I JB, even knowing how against it some people here were. I figured I could always uninstall it if I didn't like what it did to my phone's performance. Never needed to, however. The phone is as fast as it's ever been (or again, the difference in performance is so small as to be unnoticeable to me, which is what matters.)
Also, if you don't install Winterboard, you pretty much miss out on a lot.