Mavericks Server 3.0 VPN

Discussion in 'Mac OS X Server, Xserve, and Networking' started by mvmanolov, Oct 24, 2013.

  1. mvmanolov macrumors 6502a

    Aug 27, 2013
    So this has already been talked about in a few of the apple forums but lets start about it here as well....

    anyone else having issues with not being able to connect to VPN service after upgrading/clean install of Mavericks and Server 3.0...?
  2. alain651 macrumors newbie

    Nov 11, 2011
    Halifax, Canada

    I have the same problem with VPN..

    I reformat and I did a clean install of Mavericks Server twice, every thing is working (mail, websites, DNS, …) accept when i try to connect my iPad to the server using VPN… does not work.

    But i can connect using my MacBook Air, so must be an iOS issue.

    *** So hopefully Apple will resolve this problem quickly ***
  3. mvmanolov thread starter macrumors 6502a

    Aug 27, 2013
    i haven't tried using the MBA. I may do that right now. see if it works.. :D
  4. mvmanolov thread starter macrumors 6502a

    Aug 27, 2013
  5. XianPalin macrumors 6502


    May 26, 2006
    I don't have this setup myself, but I saw mentioned on another site that some people were having luck by deleting and re-installing the profile on the device.
  6. sjinsjca macrumors 68020


    Oct 30, 2008
    Sigh, L2TP VPN is no longer working on my server Mac now that I've upgraded to Mavericks and the required Server 3.0 ($20, grr).

    Both iOS and OS X devices report, "The L2TP-VPN server did not respond."

    Other services running on this machine seem to be fine. Unfortunately this is an important one.
  7. mvmanolov thread starter macrumors 6502a

    Aug 27, 2013
    Same here. I have been reading the apple discussion boards and some people seem to be having luck with running the VPN over PPTP only and ditching L2TP for the mean time until a proper fix is done.

    I would like to try that but am not sure how to go about doing that on iOS? any suggestions?
  8. sjinsjca, Oct 26, 2013
    Last edited: Oct 26, 2013

    sjinsjca macrumors 68020


    Oct 30, 2008
    PPTP is a little less secure than L2TP, but iOS supports it. Set-up is very similar to setting up the L2TP connection on those devices. Just select the "PPTP" tab.

    Having said that... I'm not having luck logging in remotely via PPTP either after enabling it on the server and restarting the VPN process. But the error message is different: instead of a timeout, I'm getting an authentication error. Maybe that's progress. I am assuming the password would be the same as for the L2TP connection so maybe that's my error. Diagnosing...

    UPDATE: I'm not seeing any server log entries for my L2TP VPN connect attempts. Still haven't figured out the PPTP authentication issues, but that seems odd..

    UPDATE: Lacking time to mess with a clearly under-tested update to OS X Server, I'm reverting to a backup from before my upgrade to Mavericks. I love Mavericks on my laptop but it clearly isn't quite ready for prime time as a server.
  9. mvmanolov thread starter macrumors 6502a

    Aug 27, 2013
    I tried PPTP as you have with the exact same results. for L2TP there are no logs either as the server is not even seeing the incoming request.. :(

    I don't have the time or patience to learn - far more than i need for what i want/need to do - how exactly the server/osx is dealing with VPN connections so i can more effectively troubleshoot.

    So i did what many are doing, entrée: Time Machine :D

    SO back to 10.8.5 and the 2.2.2 (or whatever that was) Server app. That being said my MBA is still on 10.9 and it can connect to the 10.8.5 server L2TP no problem…. so…..

    Apple what gives….? :D
  10. sjinsjca macrumors 68020


    Oct 30, 2008
    It took a little while to restore here (I'm rocking a Time Capsule with four connected USB drives for alternating backups of our various machines here-- a solution which works brilliantly) and the server is currently returned to its pre-OS X 10.9/Server 3 configuration and works without issue.

    I fired off an email via Apple's support web form ( describing the problem and requesting a refund of my $20. The VPN is mission critical here and I spent entirely too much time fussing around trying to make it work. I'd used a Linux server before, and went OS X Server precisely to avoid these opaque issues where various chicken-sacrifices and entrail-readings must be performed to get things working. I've found OS X Server to be mostly better than that, but it's hardly a panacea. Networking is not kiddie stuff, I understand, but there's little excuse for what we on this thread have gone through.
  11. mvmanolov thread starter macrumors 6502a

    Aug 27, 2013

    I completely agree!!!

    I had messed with Debian in the past but that was some years ago, and similar to you decided that severing is not something i wish to spend my life on - let alone the valuable chicken breasts that would be much better used on say a BBQ'ed shish kebob….! :D

    I also send a bug report but did not ask for a refund. Though i think i will call their enterprise support tomorrow and demand precisely that seeing as the VPN is mission critical for me too! In fact the VPN is the only reason for me buying the server app, as for me the file server is the important bit.. :D

    All of that being said - read negativity - let me insert something positive here. I am still waiting for a has well Mac Mini so i am currently running my server of my MBP and 10.9 is a really really nice upgrade performance wise. the MBP after a clean install of 10.9 was running much smoother than on 10.8.5, both the Ram management and the CPU management improvements are very welcome - and the VRAM bum to 1024 from 512 on a HD4000 was also quite nice and missed as soon as i reverted :(

    But without the VPN there is simply no point…. :( Too bad! hope a fix comes soon.

    - Missing the wave :D
  12. warplessBelmont macrumors newbie

    Oct 23, 2013
    Server 3.0 VPN Issues

    Mac Mini was running Mountain Lion Server (whatever was the most recent version) upgraded to Mavericks and now the VPN is non-responsive with the generic error:

    The L2TP-VPN server did not respond. Try reconnecting. If the problem continues, verify your settings and contact your Administrator.

    Here’s the network topology- Internet > Modem > Airport Extreme > Mac Mini via ethernet.

    For the hostname DNS we are using a dynamic DNS service, which I have verified is resolving to the machine through the router ect.

    I have tried deleting the Server App and /Library/Server as well as any pref files I could find, then rebooting, after downloading the Server App again I found all of my settings are back. Also I’ve tried removing the Server Setup Done file as well in conjunction as well as independently with no luck.

    I have tried killing raccoon via the activity monitor as well as via the command line.

    I am able to reach the machine locally via ssh and screen share, and externally via logmein.

    I have tried an iPhone 5s locally and externally, and two MacBook Airs internally and externally as well.

    I have deleted the VPN port forwarding entry in the Airport, tried putting it back manually as well as via the Server App and the drop down menu in the Airport.

    I am 99% sure the traffic is reaching the server as I can see the following when I try to authenticate to the VPN, please note this is always the same for each VPN client:

    Oct 23 08:22:10 hostname racoon[224]: Connecting.
    Oct 23 08:22:10 hostname racoon[224]: IPSec Phase 1 started (Initiated by peer).
    Oct 23 08:22:10 hostname racoon[224]: IKE Packet: receive success. (Responder, Main-Mode message 1).
    Oct 23 08:22:10 hostname racoon[224]: >>>>> phase change status = Phase 1 started by us
    Oct 23 08:22:10 hostname racoon[224]: IKE Packet: transmit success. (Responder, Main-Mode message 2).
    Oct 23 08:22:10 hostname racoon[224]: IKE Packet: receive success. (Responder, Main-Mode message 3).
    Oct 23 08:22:10 hostname racoon[224]: IKE Packet: transmit success. (Responder, Main-Mode message 4).
    Oct 23 08:22:10 hostname racoon[224]: Connecting.
    Oct 23 08:22:14 hostname racoon[224]: IKE Packet: transmit success. (Phase 1 Retransmit).
    Oct 23 08:22:47 --- last message repeated 3 times ---
    Oct 23 08:22:50 hostname racoon[224]: IKE Packet: transmit success. (Phase 1 Retransmit).
    Oct 23 08:23:10 hostname racoon[224]: IKE Packet: transmit success. (Phase 1 Retransmit).
    Oct 23 08:23:59 --- last message repeated 1 time ---
    Oct 23 08:23:59 hostname racoon[224]: IKE Packet: transmit success. (Phase 1 Retransmit).
    Oct 23 08:24:56 --- last message repeated 1 time ---
    Oct 23 08:24:59 hostname racoon[224]: IKEv1 Phase 1: maximum retransmits. (Phase 1 Maximum Retransmits).
    Oct 23 08:24:59 hostname racoon[224]: Phase 1 negotiation failed due to time up. 2194c11c97819d97:a29d73f04fe7e67f

    Rolling back to ML Server 2.2 and this works with no settings changes- something is majorly up!

    Please everyone help bring this to Apple's attention as Enterprise Support doesn't want to listen. Leave reviews in the Mac App Store, contact Enterprise Support, and submit bugs to Apple. Let's get them to acknowledge and fix this ASAP!
  13. Emilio G macrumors regular

    Jul 7, 2008
    West Palm Beach, FL
    Here's what I've found out in my travels.

    My Mavericks iMac cannot connect to a Mavericks Server and neither can an iOS 6 or iOS 7 device connect to the Mavericks Server either. The weird thing is that my personal Mavericks Server can connect to a client's Mavericks Server and vice versa with no issues. On the same note, another client's Mountain Lion Server can connect to my Mavericks Server.

    So to sum it up, previous versions of OS X and Mavericks Server to Mavericks Server connections are possible but not when trying to connect a regular Mavericks machine to a Mavericks Server nor does iOS 6 or 7 connect.

    As other have said, VPN is mission critical and I'm aggravated that there's still no word from Apple on this.
  14. eusebe, Oct 28, 2013
    Last edited: Oct 28, 2013

    eusebe macrumors newbie

    Oct 28, 2013
    It seems that changing the max socket buffer size in osx solves the problem. At least, it worked for me.
    Here are the commands. The first one gives the current value while the second one sets it to a highe value.
    sysctl -a|grep maxsockbuf
    sudo sysctl -w kern.ipc.maxsockbuf=1000000

    Edit: worked only once here. Tried with higher values, no success.
  15. mvmanolov thread starter macrumors 6502a

    Aug 27, 2013
    i;ve seen this posted before, but have not tried it. I don't have the time to reinstall and try now so can you tell us exactly what your setup was and what the problem was for this fix to work?
  16. ratsg macrumors 6502

    Dec 6, 2010
    here is my output from a 10.6.8 box.

    root# sysctl -a | grep maxsockbuf
    kern.ipc.maxsockbuf: 4194304

    I am somewhat surprised that my default value is 4 times greater than your increased value.
  17. scheming macrumors member

    Mar 15, 2005
    Just did a clean install of Mavericks on my MacBook Pro, Late 2008. I skipped signing in to iCloud or any other services and avoided made no other changes to settings. I installed Server 3.0 and configured only VPN for L2TP and PPTP. Problem persists. A packet trace on the client device shows the device is indeed receiving the IKE Phase 1 packet from the server, however it appears to ignore it or otherwise finds it to be invalid.

    The strange thing that I have found is what is in /private/etc/racoon/psk.txt file:

    # IPv4/v6 addresses
    #	asecretkeygoeshere
    #	asecretkeygoeshere
    # 3ffe:501:410:ffff:200:86ff:fe05:80fa	asecretkeygoeshere
    # 3ffe:501:410:ffff:210:4bff:fea2:8baa	asecretkeygoeshere
    # macuser@localhost	somethingsecret
    # FQDN
    # kame		hoge
    First, I obviously did not make the pre-shared key "asecretgoeshere" and the other IPs in this file do not refer to my network at all. It is my understanding, based on the information in /private/etc/racoon/racoon.conf, psk.txt is the file that is looked at when checking for the PSK. Every line in the PSK file is commented out with #. So what is going on here?
  18. mvmanolov thread starter macrumors 6502a

    Aug 27, 2013
    can you please try what eusebe (above) is suggesting and report back?
  19. warplessBelmont macrumors newbie

    Oct 23, 2013
    Spoke to Apple Enterprise Support

    Spoke to Apple Enterprise Support this morning and they are aware of the issue now. We spent about 2 hours troubleshooting and trying everything the tech could think of, in the end he gathered logs from my server. At this point they are leaning towards an issues with NAT and Mavericks Server. They're working on it, most likely be addressed in an update to the Server app. Just wanted to share.
  20. scheming, Oct 29, 2013
    Last edited: Oct 29, 2013

    scheming macrumors member

    Mar 15, 2005
    I already restored from Time Machine Backup. Sorry I didn't see your request to run that command in terminal beforehand. This is what I see right now:

    Gerus-MacBook-Pro:~ geru$ sysctl -a|grep maxsockbuf
    kern.ipc.maxsockbuf: 4194304
  21. mvmanolov thread starter macrumors 6502a

    Aug 27, 2013
    no worries,

    I think i'll wait for the official patch before i go back to Mav. VPN is mission critical for me. in fact its the only reason for me to get the server app. i'll call enterprise support today and ask for a refund cue this is sort of ridiculous...
  22. Voch macrumors member

    Jan 27, 2006
    VPN first timer...

    Thanks for this thread. I'm trying OS X Server on my Mac mini for the first time to add CardDAV and CalDAV services because of Apple's removal of Info syncing in the new iTunes; those features are working wonderfully and I don't know how I lived without them before. I'm adding VPN for myself as an additional perk of purchasing and am having the same issues shown here.

    As an aside and if it helps anyone debug their configurations, I *can* get VPN to work within my LAN (connecting to MachineName.local instead of my outside DNS name), proving that I have it configured correctly within and on my clients (MacBook Pro and iPhone 4S). But, yeah, VPN within the LAN is useless (it's for exposing my network outside...duh).

    And I love that the automatically configures the ports of my AEBS. I assume it's doing that part correctly for me (UDP 500/1701/4500 and TCP 1723)?
  23. thorsten macrumors newbie

    Jan 18, 2008
    VPN only works from OS X, not iOS

    I can perfectly connect to Server 3.0 from my MacBook Air through L2TP from within the network and from outside. Server VPN log shows all the details.
    But when trying to connect from iOS and even from my Android, there is not a single entry in the Server logs that anything even reaches the server.
    I checked using TCPMon and using PPTP and was clearly showing that packages were forwarded to the server. But still nothing showed in the log.
  24. mvmanolov thread starter macrumors 6502a

    Aug 27, 2013
    so, i just saw this on the Apple discussion boards:

    "Hello there as well,

    I've the same issue and I investigate the problem. The reason why it does not work is, that the racoon (IKE Daemon) does not accept connections on port 4500 (IKE for NAT-T) if the source port is random generated.

    Since Mavericks and IOS7 the source port from the client is no longer 4500, this lead to this problem (except you have a old VPN connection already setup bevor you update to IOS7 on your Phone).

    If you are in the same network like your server, the IKE NAT-T is not used. In this case the regular port 500 (IKE) is used, and this works as expected. At the moment we have to wait if the problem is fixed by Apple.

    There are two possibilities, they can adjust the clients or the server configuration. However if you want to use VPN with OS X native methods, use PPTP. This is not affected but of course it provides no Layer 2 Tunneling.



    Hope this helps someone!
  25. joergy macrumors newbie

    Oct 31, 2013
    VPN from 10.7.5 doesn't work either

    sysctl -a|grep maxsockbuf gives me

    and the log shows

    I don't find a way to switch from L2TP to PPTP on 10.7.5 (client), so I couldn't check this out...

    Any new experience here?

Share This Page