It really is just all relative to what was being done on personal computers at the time. When I bought my dual 1.8GHz G5 in 2004, it was a $2000 computer, before the $500 graphics card or ~$300 of RAM was thrown in. And for the time, it screamed--it blew through Flash MX development, Photoshop CS2 renders, and could play the original WoW really smoothly (keep in mind this is when the game was new). Heck, a dual G5 with a Geforce 6800 or ATI X800 was the only Mac that could run Halo smoothly on high settings for quite some time. Keep in mind this machine shipped with Panther, and was best suited for Tiger--both of which were much less demanding than anything out there today.
The G5s were never great home computers in the sense of being so hot/power hungry, but for sheer performance, they were a very solid value for the first couple years ('03,'04). By 2005 they'd hit a roadblock for speed/value/heat vs what Intel could do, but if you took a dual G5 from 2004 and put it up against any Intel home computer of the same age, it'd do just fine. As for your concerns about speed of video playback...consider that Youtube didn't even exist back when the G5 was the hot new thing. Kind of a funny thought, no?