I don't have my Mac just yet so can't test the Mac version. But have downloaded the Windows version of the server and viewer.
Installed the server on my PC desktop and tried to log in with my laptop and I get an error message. I opened up my Firewall to accept connections but it still doesn't work. I haven't gone any further than that.
I'll definitely report back about the Mac version when I get my Mini.
When using VNC - try Chicken of the VNC, seems to be a little easier to configure. Try connecting in the form of your IP address to the outside world and don't forget the port # of the viewer (usually 5900) Make sure that you you have a password enabled on the server as that is usually why it disconnects.
I am by no means an expert on this, you can search osxhints.com for some answers as well, like I did at first.
Don't give up, once you understand how to connect the possibilites are endless!
I use VNC on my pocketPC to look at my home computer and to run my powerbook ala Salling Clicker.