OS X Server, hostname ambiguity, and VPN/SSH issues

Discussion in 'Mac OS X Server, Xserve, and Networking' started by mbarriault, Aug 1, 2012.

  1. mbarriault macrumors regular

    Dec 3, 2008
    With OS X Server now $20, I decided to install it on my Mac mini home server. As a test of the new services it provides, I decided to enable VPN, but thus far have been unable to connect to it. Here is my setup:

    The Mac mini is running 10.8 Server with VPN enabled. Originally, it was configured to use machinename.local for hostname, but I configured the hostname to be domain.com, a domain I've had for some time but didn't use.

    The mini is on a network provided by an Airport Extreme, which forwards necessary ports to the mini. External SSH (forwarding a nonstandard port outside to 22 inside), Minecraft server port, and web server ports all work without issues. The Airport provides DHCP services for devices connected to it.

    The Airport gets internet access via ISP modem/router, configured to use the Airport as DMZ. NAT is enabled.

    However, these are my issues
    • Airport complains about Double NAT, however disabling NAT on the modem makes external connections completely unresponsive.
    • Configuring the modem to use Advanced DMZ causes it to crash at least once a day.
    • I can no longer SSH to the mini using the machinename.local address.
    • Airport is configured to forward UDP 500, 1701, and 4500, and TCP 1723, but any incoming VPN connection fails (tested using iPhone and MacBook Air, error says server didn't respond). nmap shows all ports are filtered, but says the same about other ports that are forwarded correctly. VPN is set to default settings, except for shared secret (which I changed from a random string to help prevent typos) and VPN host name (to external domain). I've not configured any certificate anywhere yet.
    • I've also attempted using Private network (machinename.private) with no success.
    • The only relevant bit I've been able to find on this is that Back to my Mac can interfere with VPN services, but disabling it made no difference.

    If there's any help or insight anyone can provide, I would greatly appreciate it.
  2. mbarriault thread starter macrumors regular

    Dec 3, 2008
    Turns out the problem was indeed with BTMM, which runs on the same protocols as OS X Server's VPN service, in fact Apple explicitly recommends against trying to use both. When I tried disabling BTMM last time, my firewall wasn't configured properly.

Share This Page