The single 1ghz FW800 model has a bus speed of 133mhz, which means that you won't get any benefit from faster RAM. With that said, in my experience, PC-3200 is easier to find than the slower speeds, and will generally downclock fine. You want 512mb low density sticks(16 chips) to max it at 2gb. So, in other words, use faster RAM, but any performance improvements you see will be from having more RAM, not faster.
There are a variety of both bootable and non-bootable SATA cards that will work. Hopefully others can provide the specifics. I have a few Sonnet bootable ones, but these were always expensive and now bring silly money on the secondary market. A popular option for a while was to buy a certain chipset and flash it to be Mac bootable, although the last I checked the cards that would flash easily seemed to have dried up.
The most available ADC-compatible upgrade is the Radeon 9000, although in my experience it's debatable as to whether or not this gives you any real improvement, especially in OS X. The "best" plug and play ADC card for this system(especially if you don't need OS 9, which these didn't officially support although some very smart folks have now figured out how to do it) is the Radeon 9700, but these are quite uncommon. I'm also rather partial to the Geforce 4Ti, although in my experience a lower end CI card(like a Radeon 9600) edges out the 4Ti in OS X.
If you're not averse to modding things a bit, I like the Radeon 9600 from the G5 in 4x AGP G4 systems(Digital Audio on up). You need to tape/disable pins 4 and 7 and do a bit of soldering to get ADC power working, but I ran one in my Quicksilver for several years. This is also not a 100% compatible solution-the USB ports on the display will work, but power on via the monitor does not work.
If you don't mind using an A1006 to power your Cinema display, you have a few other good options. The easiest plug and play one is probably the Radeon 9600 PC & Mac. The Geforce 6200 is good for flashing(you can flash 5200s also, but this is a crummy enough card that I wouldn't bother). I'm not well versed in Radeon 9800s, although I have one in a Sawtooth-IIRC there are some specific things you need to look for to ID a good candidate for flashing. Also, I think mine has a double-height cooler, and this is desirable since these cards can cook themselves pretty badly.
A few years ago, ATI FireGL X3s, which flash nicely into Mac X800XTs, were plentiful on Ebay. Unfortunately, these seem to have dried up. They were hit or miss in the Quicksilver/DA(I have had a few of them, and have the only one I have personally known to work in Digital Audio) but tend to work fine in the MDD. This is one of the highest performing cards that will work in G4 towers.
USB 2.0 cards that are compatible with OS X are not difficult to find. IIRC, I've always had good luck with Via chipsets. I've run a few USB 2.0/FW combo cards. Of course, buy Sonnet if you want guaranteed compatibility, although like I said I've had good luck in this category with generic cards without any finagling-as with many things Mac related it comes down to just finding the correct chipset and it will work.
Unfortunately, I don't think anyone has identified a PCI "N" compatible card. There is a specific model of Motorola card whose number I don't have handy that works for a G card, although personally I wouldn't bother with it in a FW800 since it basically gives you the same capabilities as an Airport Extreme card and is more difficult to find.
