I guess I am limited to 10.5.8 after all - 10.6.8 A5 doesn’t boot, instead it displays a gray background with an instruction to turn off my Mac
Which image did you use and how specifically? Are there any PCIe cards (aside of the GPU) installed?
Did you try booting into boot picker (with Alt pressed)?
It is not impossible that some specific configuration is unsupported, but I had no boot issues on G5 2.3.
Generally safe procedure would be to download the image from the Mac (Windows can screw dmg), clone it to a different partition or drive with Disk Utility with Erase destination checked. Boot with Alt pressed, choose the boot drive.