I would suggest trying to restart your wireless access points/routers. Sometimes if there are things competing for bandwidth it'll create a detrimental effect.
So if you're downloading a bunch of free TV episodes on your Mac/PC, and you've got multiple connections for things like your Television, AppleTV, Wii/XBox/PS3, Roku Box, etc, you're all competing with bandwidth.
There's only so much you'll be able to sustain, and typically the upload speed on cable is capped around 2 megs on any provider (some are more or less, and there are variants, for example my basic ISP internet service is 2 megs capped upload, but I can upgrade to another tier of service for another $10 per month and I think it jumps to 5 megs up and something crazy like 30 megs down.
I'd also see if this issue is happening over EVERY wifi spot you're using. This could narrow down the cause to either your router or the other person you're FT'ing with. Next time you're both in the same location, use the same router and FT and see how it works. If it's bad, might point to your router.