Could there be other wireless networks nearby using the same channels as you are?
Thus, resulting in interference?
You can check this with the utility "iStumbler".
I believe there are some others than can show you the same information.
Also:
If you are having problems while connected at 2.4ghz, have you tried switching over to 5ghz?
Or, vice versa?