Ok, that's not characteristically bad Flash performance, that's abnormal. I get playback like that on an 8-year-old dual 1GHz G4, and any Intel-based Mac should be substantially better than that.
Here's a question: Do you have OTHER pages open at the same time? Thing with Flash is--and this is the main reason I hate it so much, not the video performance--that any badly written Flash object can easily consume 100% CPU, and as I said before, Flash isn't multithreaded. And when you're talking about Flash-based web ads, "badly written" means "everything."
Which is to say that any time you have a page with a Flash ad on it open--which is practically any time you have a page open--that ad is probably using 100% of a CPU core to do whatever stupid thing it's doing. It's so bad--hearing the fans on my MBP spin up and feeling the bottom heat significantly--that FlashBlock is now the first thing I install on any Mac I'm using.
Now, maybe this isn't the cause of your problem, but I could imagine that if there's another Flash file in the background while you're trying to watch 1080p video eating up CPU, it could make it a lot choppier.
Not being a regular Firefox user I can't tell you what the equivalent of Flashblock is for it, but you might try, for the heck of it, quitting Firefox, opening it, going directly to Youtube with nothing else open, and seeing if the performance is any better.
I'm trying very, very hard to resist derailing this thread on the pro/anti-Flash argument, so I'm not going to respond to this in any detail. I will say, however, that Windows does NOT have 90% marketshare in the mobile space--on the contrary, MS has about 15% to Apple's 25%, and when you look at the amount of actual
web TRAFFIC, meaning how many people will actually care one way or the other about whether they can watch your video, the numbers are in the low 40% range for the iPhone, a hair less for Android, and a whopping 2% for Windows Mobile. The situation outside the US is even more stark, with the iPhone accounting for about 50% and the nearest competitor, Android, at about 25%.
I think there's a legitimate argument to be made against Flash in general, but one way or the other saying that Apple's lack of support for it on the iPhone will hurt them more than Adobe is questionable at best at this point in time.
There's also the important fact that Flash as a video distribution medium is just a middle-man--it's feeding you the same compressed MP4 video stream that the OS could be handling itself or handing off to the h264 hardware decoder in most phones. That YouTube is moving strongly toward HTML5 video does not bode well for Flash for video distribution.