How many page swaps are you seeing with 2GB in the machine right now?
If you are seeing quite a few it will slow the machine down -- basically to the speed of the G4 (drive access generally the same between the G4 & G5).
The testing showed that the G5s do need quite a bit of memory to give you the full benefit of the new architecture, but I don't remember anybody really playing around with the memory and apps to see how BIG the benefit is.
Just the offhand note that there was a boost in the benchmarks when xGB of memory was used.