Maybe I just didn't configure it properly, but I was using CotVNC to connect to a windows box connected to the same switch as my Mac Pro, and it ran slow as heck. I could see the mouse moving on the other windows display almost realtime, but it took a couple seconds to update on my Mac... I tried adjusting the settings, and it only got worse... Even with going to 256 colors. :/
I ended up just running TightVNC client on windows in my VMWare instance. Everything moved close to realtime.
I should also add the VNC server on windows was TightVNC as well... CotVNC did give the option of what protocol to use, and I disabled all but tightvnc's, and still no improvement...