Wirelessly posted (iPhone 3GS: Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_3_5 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8L1 Safari/6533.18.5)
If you're behind a hardware firewall such as any router available, you don't need to enable the software OS X firewall. It doesn't hurt to have it on, but it's not necessary.
If you go and use public WiFi or another network, you'd be wise to enable it.
Either way, having it enabled shouldn't affect your speed so much where you can notice it.