I bought the modded 9600 Pro card from OWC for my Dual 533. The modifications involve removing the conducting material from 2 leads on the card edge connector to reduce the speed of the bus from AGP 8x to 4x, since G4 towers can't use an 8x card. The ADC connector is non-funtional because the physical location of the ADC power connector for AGP cards is in a different location on G5 motherboards, so the connectors don't line up. You can make it work with a little elbow grease. There is a website that describes the modification in detail with pictures right
here.
The GeforceFX5200 is currently not usable in a G4 (and there's no point really, since the Radeon 9600 is a better all around performer).
Don't bother with the Geforce4Ti, they are way overpriced due to their rarity, and their performance is not good enough to justify that price - they often go for as much as a 9800. Other than the 9600/9800, your other options are the Radeon 7000, 7500, 8500, 9000, 9700 or Geforce3 or 4MX.
I did a little research when I bought my card, and found that the Geforce4MX was actually slower than the Geforce3 in most benchmarks, so I'd avoid the Geforce4MX. The Geforce3 is the oldest card I'd be willing to use - you can get them pretty cheap off ebay, though many of them are flashed PC cards (not that that is a bad thing necessarily).
The Radeon 9000 is a decent performer, but it costs slightly more than the modded 9600, which kicks the 9000's butt in gaming performance. Only buy it if you need to use two displays and can't afford the 9800. Avoid the 7000. Ditto the 7500. The 8500 is close in performance to the 9000, but somewhat rare. The 9700 is pretty much as rare as the Geforce4ti and will command a similar price so again give it a pass unless a deal shows up.
EDIT: There is also the Radeon 9200; it is a PCI card, not AGP and is generally slower than the 9000. Avoid.
Finally, the 9600, 9700 and 9800 fully support Core Image, the upcoming feature of OS 10.4 which offloads a lot of OS graphics processing to the video card. This should speed up everyday tasks like opening windows and menus and whatnot. Other cards are not fully supported and won't benefit as much or at all.
Sorry for the long-windedness of the post, but I did a lot of looking before I upgraded my video card and I came to the conclusion that the 9600, 9700 and 9800 are the only cards that one should be looking at to buy. Of these, the (OWC-modded) 9600 Pro is BY FAR the cheapest and is pretty close in performance to the more expensive cards for most tasks on a G4. The modding should not affect the card's performance or reliability and for ~$120 it's a steal.