Mighty Mouse Issue when using browsers....

Ok.....this is kinda weird. It seems that my mouse intermittently goes crazy, like automatically going back a page or automatically scrolling down to the bottom of the page and not allowing me to scroll back up. It only happens in the browsers......currently using Safari and Firefox. If I go out to another application and come back to a browser without necessarily having to quit the browser itself, it clears up for a bit but then starts acting up again.

I've heard of others here having issues with their mouse, but if it was a hardware issue, wouldn't you think it would also happen in the system windows and not just the browsers ?