The iMac G5 is a nice machine. Just traded mine away, though, as the noise bothered me. I don't recommend them to my friends and family, however, because of both the noise (this still isn't 100% fixed in RevB models - go read the Apple support forums) as well as the very real chance of premature HDD (and other component) failure due to the high heat levels. I thnk barefeats.com has a good article about the heat on the HDD of the RevB (and A) iMac G5.
Also, after having one, I tend to steer people clear of all-in-one systems, except laptops IF they need portability. Seems like a better investment, to me, to spend a little more now, and then be able to spend less on future upgrades. Especially if you decide to keep the old computer for some other use rather than 'trading up'.
Rather than go with a used PB, I would consider a new 12" iBook. There should be an update to them soon and I would guess that they'll be getting a Core Image capable GPU. You can snag a nicesly loaded 12" iBook, a nice LCD, and a high speed FW HDD for about $1500, especially if you can hold out untill the next revision. You could also grab a current generation iBook on clearance after the new ones come out and save some cash.
If you don't absolutely need a portable system, go for a mini with a nice LCD and extra RAM and external HDD. Later on you can move up to a PowerMac if you want more power, or you could move to a laptop if you decide you want portability, and keep your external devices, etc.