One major problem is that Apple is married to Motorola because of the Altivec. If there were two competing chip makers that both made equally or similarly capable chips for Apple to choose from, development would be far faster due to each company attempting to gain an advantage.
Here, we have IBM making great chips, but unable to compete with Motorola because it has a patent on its altivec technology. Thus, because Apple has designed its machines to really make use of Altivec, it's stuck.
Is it Motorola's fault that it has designed a chip that Apple has handcuffed itself to or Apple's fault for snapping on the cuffs?
Motorola has almost no incentive to innovate in its PowerPC development because it has no competition. IBM can't develop a version of Altivec because of the patent, so Apple's got problems.
I see as solutions Apple attemption to optimize its systems for the non-altivec PowerPC chips made by IBM or even, dare I say, x86 chips (or the newer 64 bit(?) chips set to come out) so that it has alternatives.
If it chooses not to do that, then it's doomed to suffer the same fate as your stereo system. It is limited by its least capable component. In this case, the chip.