How To Properly Setup VPN Server? (Not Working)

Discussion in 'Mac OS X Server, Xserve, and Networking' started by mohammad, Oct 27, 2013.

  1. mohammad macrumors member

    mohammad

    Joined:
    Feb 7, 2008
    Location:
    Netherlands
    #1
    I just bought OS X server (Maverickes) with the target of being able to setup a VPN Server on my local Macbook Pro laptop. I have done the required steps but I'm not able to do a VPN from my iPhone to my VPN server.

    My objective is that I want to be able to VPN from any device to may Macbook Pro mainly for internet browsing, mail.

    Here is what I have done:

    1- Installed OS X server on my Mabckook Pro
    2- They only services that is set to ON is VPN
    3- Here are the settings for VPN:
    • Status is green and it says 'Available at macbookpro.lan'
    • Configure VPN for: 'L2TP'
    • VPN Host Name: I have provided the IP address of my Macbook Pro that it gets form my local internet router, it does have green status
    • Shared Secret: I have given a shared secret
    • Client address: allowed 5 devices
    • DNS setting: One question I have is what DNS should I provide here? My internet router's DNS? or my internet routers's default gateway?
    • Routers: Do I need to provide any routes?
    4- I have created a new test account on my Macbook Pro to be able to use it for the VPN through my iPhone
    5- Now I go to my iPhone and setup VPN as follows:
    • Server: I provide my public IP address that I get from www.google.com
    • Account: I enter the user name of the account I just created in my Macbook Pro
    • RSA SecurID: is set to off
    • Password: I provide the password of the account I created on my Macbook Pro
    • Secret: I enter the shared secret key
    • Send All Trafic: is set to On

    I have also forwarded the following ports from my internet router to my Macbook Pro's local IP address:

    Protocol Port Range
    TCP 1723 - 1723
    UDP 500 - 500
    UDP 1701 - 1701
    UDP 4500 - 4500

    When I set VPN on on my iPhone I get the following error: "The L2TP-VPN server did not respond..."

    Here is my VPN service log:
    #Start-Date: 2013-10-27 15:16:26 CET
    #Fields: date time s-comment
    2013-10-27 15:16:26 CET Loading plugin /System/Library/Extensions/L2TP.ppp
    2013-10-27 15:16:26 CET Listening for connections...

    And here is my System Log when I do a VPN from the iPhone:
    Oct 27 15:41:09 macbookpro.lan racoon[3222]: >>>>> phase change status = Phase 1 started by us
    Oct 27 15:41:09 macbookpro.lan racoon[3222]: IKE Packet: transmit success. (Responder, Main-Mode message 2).
    Oct 27 15:41:09 macbookpro.lan racoon[3222]: IKE Packet: receive success. (Responder, Main-Mode message 3).
    Oct 27 15:41:09 macbookpro.lan racoon[3222]: IKE Packet: transmit success. (Responder, Main-Mode message 4).
    Oct 27 15:41:09 macbookpro.lan racoon[3222]: Connecting.
    Oct 27 15:41:12 macbookpro.lan racoon[3222]: IKE Packet: transmit success. (Phase 1 Retransmit).
    Oct 27 15:41:46 --- last message repeated 3 times ---
    Oct 27 15:41:46 macbookpro.lan racoon[3222]: IKE Packet: transmit success. (Phase 1 Retransmit).
    Oct 27 15:42:04 macbookpro.lan racoon[3222]: IKE Packet: transmit success. (Phase 1 Retransmit).
    Oct 27 15:42:49 --- last message repeated 1 time ---
    Oct 27 15:42:49 macbookpro.lan racoon[3222]: IKE Packet: transmit success. (Phase 1 Retransmit).
    Oct 27 15:43:46 --- last message repeated 1 time ---
    Oct 27 15:43:46 macbookpro.lan racoon[3222]: IKEv1 Phase 1: maximum retransmits. (Phase 1 Maximum Retransmits).
    Oct 27 15:43:46 macbookpro.lan racoon[3222]: Phase 1 negotiation failed due to time up. dfec20fdf9615471:b34d569a7e265609

    I would appreciate if you tell me what am I doing wrong?
     
  2. mvmanolov macrumors 6502a

    Joined:
    Aug 27, 2013
    #2
  3. mohammad, Feb 1, 2014
    Last edited: Feb 1, 2014

    mohammad thread starter macrumors member

    mohammad

    Joined:
    Feb 7, 2008
    Location:
    Netherlands
    #3
    I have now got the latest OS X Mavericks VPN fixes installed, but I'm still unable to VPN from my iPhone to my OS X server.. I'm now getting the following error in my OS X server VPN log... what am I doing wrong?

    2014-02-01 23:52:04 CET Incoming call... Address given to client = 192.168.3.226
    2014-02-01 23:52:04 CET --> Client with address = 192.168.3.226 has hung up


    And this is a dml from TCPDUMP


    00:05:38.957807 IP 84.xxx.xxx.242.37437 > macbookpro.lan.isakmp: isakmp: phase 1 I ident
    00:05:38.958671 IP macbookpro.lan.isakmp > 84.xxx.xxx.242.37437: isakmp: phase 1 R ident
    00:05:39.268785 IP 84.xxx.xxx.242.37437 > macbookpro.lan.isakmp: isakmp: phase 1 I ident
    00:05:39.274870 IP macbookpro.lan.isakmp > 84.xxx.xxx.242.37437: isakmp: phase 1 R ident
    00:05:39.369090 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: NONESP-encap: isakmp: phase 1 I ident[E]
    00:05:39.369912 IP macbookpro.lan.ipsec-msft > 84.xxx.xxx.242.37444: NONESP-encap: isakmp: phase 1 R ident[E]
    00:05:39.370310 IP macbookpro.lan.ipsec-msft > 84.xxx.xxx.242.37444: NONESP-encap: isakmp: phase 2/others R inf[E]
    00:05:40.185232 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: NONESP-encap: isakmp: phase 2/others I oakley-quick[E]
    00:05:40.186126 IP macbookpro.lan.ipsec-msft > 84.xxx.xxx.242.37444: NONESP-encap: isakmp: phase 2/others R oakley-quick[E]
    00:05:40.348325 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: NONESP-encap: isakmp: phase 2/others I oakley-quick[E]
    00:05:40.348329 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: UDP-encap: ESP(spi=0x0d3ef4d3,seq=0x1), length 132
    00:05:41.414004 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: UDP-encap: ESP(spi=0x0d3ef4d3,seq=0x2), length 132
    00:05:43.257222 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: UDP-encap: ESP(spi=0x0d3ef4d3,seq=0x3), length 132
    00:05:47.250902 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: UDP-encap: ESP(spi=0x0d3ef4d3,seq=0x4), length 132
    00:05:51.185552 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: UDP-encap: ESP(spi=0x0d3ef4d3,seq=0x5), length 132
    00:05:55.545039 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: UDP-encap: ESP(spi=0x0d3ef4d3,seq=0x6), length 132
    00:05:59.231882 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: UDP-encap: ESP(spi=0x0d3ef4d3,seq=0x7), length 132
    00:05:59.846227 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: isakmp-nat-keep-alive
    00:06:00.767580 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: NONESP-encap: isakmp: phase 2/others I inf[E]
    00:06:00.767664 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: NONESP-encap: isakmp: phase 2/others I inf[E]
     
  4. noremacyug macrumors member

    Joined:
    Feb 5, 2014
    #4
    How To Properly Setup VPN Server? (Not Working)

    Did this ever get resolved? I too am trying to setup the vpn on my mini running mavericks with osx server, latest updates installed. I've setup the vpn before on my router with no issues, but for the life of me I cant get my iphone to connect to my mini's vpn. I have forwarded the aforementioned ports and have a dyndns account. I have no issues connecting to other services on my mini from the wan. I could easily be doing something wrong, but I don't know what it would be, it looks pretty straight forward.
     
  5. mvmanolov macrumors 6502a

    Joined:
    Aug 27, 2013
    #5
    the latest server app does not have these issues, it was fixed a while back with 10.9.2
     
  6. noremacyug macrumors member

    Joined:
    Feb 5, 2014
    #6

    I don't know what I'm doing wrong then.
     
  7. noremacyug macrumors member

    Joined:
    Feb 5, 2014
    #8
    thanks, but i'd like to try to make it work as is without any added apps. i even put in my ip address both server and client side. still no joy. the password and secret key match up, the ports are forwarded..... what gives? why won't this work? i almost have to be overlooking something. i really wish that apple had some better documentation on this. even though it looks simple, this has been a headache.
     
  8. mus0r macrumors regular

    Joined:
    Mar 27, 2005
    #9
    Here's what you do.

    Google search Asus (or whatever brand your prefer) routers with VPN built-in. Buy it. Enjoy a grief-free VPN.

    I gave up on Apple's VPN service a long time ago.
     
  9. noremacyug macrumors member

    Joined:
    Feb 5, 2014
    #10
    Yeah. The vpn on my router works fine (asus rt-ac66u) It's more of a personal quest to get the vpn in my Mac working now.
     
  10. noremacyug macrumors member

    Joined:
    Feb 5, 2014
    #11
    Time to resurrect this.

    So, just a few minutes ago I tried connecting to my home vpn from my MBA whilst away from home, worked perfectly. However, when plugging those same credentials into my iphone, it won't connect. What gives?
     
  11. Alrescha macrumors 68020

    Joined:
    Jan 1, 2008
    #12
    Depending upon how your router deals with IPSec, "a few minutes" may not be long enough for your router to forget about your Macbook. It may only be able to deal with one IPSec tunnel at a time (it would not be the first).

    A.
     
  12. noremacyug macrumors member

    Joined:
    Feb 5, 2014
    #13

    I have an ipfire box built for routing/firewall. The vpn server is on my Mac mini. I simply cannot get my phone to connect. I can connect using my MBA tethered to my phone however, which I know isn't apples to apples. But at least I know att isn't blocking ports or whatnot. It's really annoying.
     
  13. satcomer macrumors 603

    satcomer

    Joined:
    Feb 19, 2008
    Location:
    The Finger Lakes Region
    #14
    I also came up with a video for 10.9.x Server VPN called Mavericks Server Part 16: VPN:



    This could help in setting up VPN in OS X 10.9.x Server.
     
  14. noremacyug macrumors member

    Joined:
    Feb 5, 2014
    #15

    Yeah, I've already seen his video. I have it all set like he shows and it will work with my MBA, just not my iphone. I don't get it.
     
  15. satcomer macrumors 603

    satcomer

    Joined:
    Feb 19, 2008
    Location:
    The Finger Lakes Region
    #16
    What service to you use for the domain? Did you go with No-IP? What do you use?
     
  16. noremacyug macrumors member

    Joined:
    Feb 5, 2014
  17. mwb, May 7, 2014
    Last edited: May 7, 2014

    mwb macrumors newbie

    Joined:
    Jul 21, 2011
    #18
    Have you made sure Back to My Mac is turned off?

    Edit: Never mind, I see you can already VPN from your MBA.
     
  18. mohammad thread starter macrumors member

    mohammad

    Joined:
    Feb 7, 2008
    Location:
    Netherlands
    #19

    Thanks for the videos I looked at them and tried setting them as you have shown with .private but I still cannot connect with my iPhone to my MacbookPro which is running the server..

    I always get the following error

    2014-02-01 23:52:04 CET Incoming call... Address given to client = 192.168.3.226
    2014-02-01 23:52:04 CET --> Client with address = 192.168.3.226 has hung up

    What does this error actually mean?
    What should I do next? :-(
     
  19. AppleNinja88 macrumors newbie

    Joined:
    May 12, 2014
    #20
    Things to check

    Did you save a server VPN configuration profile and load it on your iPhone? This would probably help you with a lot of the problems your having on your iPhone. What kind of router do you have? You can control Apple Airport Routers with the Server app which open the correct VPN ports. If you can connect from your MBA then there is probably a setting your missing in your iPhone(referring back to the VPN configuration profile. Also, Where are you connecting from when your on your iPhone? Wifi or Cellular network? I would try a wifi connection first (other than your house duh... a good one too, not starbucks or some lame **** like that) Connecting over a cellular network is kinda spotty depending on service. I have also noticed when connecting over my iPhone's cellular network that you have to attempt the connection a couple times before it connects sometimes, are you trying this more than once or when it fails once or do you panic and then try to figure out if there is a problem. Do you have a static IP address from your ISP? or are you using a service that is updating the IP for a domain name automatically? You can find out what your outside ip address is by typing whats my ip in google, you most likely do not have a static IP. Hope some of this helps. Let me know how it goes buddy.
     
  20. mohammad thread starter macrumors member

    mohammad

    Joined:
    Feb 7, 2008
    Location:
    Netherlands
    #21
    Hi, I had never tried saving the VPN configuration profile and then loading that onto my iPhone but I still cannot connect and get the same message :-( I do not have a static IP but the dynamic IP that I get from my ISP does not change (as far as I know and have checked)

    Nevertheless I started this thread and at the top I did explain how I have set things up, but appreciate if you can help me out on this. Below is the details:

    1- Installed OS X server on my Mabckook Pro, I have tried both .lan and .private for using VPN (I haven't tried using a domain name yet)
    2- They only services that was set to ON is VPN
    3- Here are the settings for VPN:
    Status is green and it says 'Available at macbookpro.private'
    Configure VPN for: 'L2TP'
    VPN Host Name: I have provided the IP address of my Macbook Pro that it gets form my local internet router, it does have green status
    Shared Secret: I have given a shared secret
    Client address: allowed 5 devices
    DNS setting: One question I have is what DNS should I provide here? My internet router's DNS? or my internet routers's default gateway?
    Routers: Do I need to provide any routes?
    4- I have created a new test account on my Macbook Pro to be able to use it for the VPN through my iPhone
    5- Now I go to my iPhone and setup VPN as follows:
    Server: I provide my public IP address that I get from www.google.com
    Account: I enter the user name of the account I just created in my Macbook Pro
    RSA SecurID: is set to off
    Password: I provide the password of the account I created on my Macbook Pro
    Secret: I enter the shared secret key
    Send All Trafic: is set to On

    I have also forwarded the following ports from my internet router to my Macbook Pro's local IP address:

    Protocol Port Range
    TCP 1723 - 1723
    UDP 500 - 500
    UDP 1701 - 1701
    UDP 4500 - 4500

    When I use the VPN on on my iPhone I get the following error: "The L2TP-VPN server did not respond..." and in the VPN job I get the following '2014-05-18 21:12:34 CEST --> Client with address = 192.XXX.XXX.XX has hungup'
     
  21. bilbo--baggins macrumors 6502a

    Joined:
    Jan 6, 2006
    Location:
    UK
    #22
    I got it working

    I followed the youtube video given in this thread, and at first I got the same message about the server not responding.

    I changed 2 things

    1) Instead of entering the settings on the iPhone, I emailed the Configuration Profile to my iPhone
    2) After installing it on the iPhone, I then replaced the Shared Secret (using 1Password).

    To answer your questions about DNS:
    I specified 2 forwarding servers - which are OpenDNS addresses
    My router has my servers IP address for DNS (the router is doing DHCP, so presumably it gives out my servers IP address to other devices on the network).
    In terms of setting up DNS in Mavericks Server, I let it do the set up for me.

    For reference (though I don't think it matters too much) my VPN hostname is server.imac.private and this gets put into the Configuration Profile, so on the iPhone I replaced this with my external IP address (found by googling What is my IP).

    I have no routes configured.

    I opened up the relevant ports on my router, which are the ones you listed.
     
  22. Iamgort macrumors regular

    Joined:
    Aug 27, 2011

Share This Page