How To Properly Setup VPN Server? (Not Working)

mohammad · Oct 27, 2013

I just bought OS X server (Maverickes) with the target of being able to setup a VPN Server on my local Macbook Pro laptop. I have done the required steps but I'm not able to do a VPN from my iPhone to my VPN server.

My objective is that I want to be able to VPN from any device to may Macbook Pro mainly for internet browsing, mail.

Here is what I have done:

1- Installed OS X server on my Mabckook Pro
2- They only services that is set to ON is VPN
3- Here are the settings for VPN:

Status is green and it says 'Available at macbookpro.lan'
Configure VPN for: 'L2TP'
VPN Host Name: I have provided the IP address of my Macbook Pro that it gets form my local internet router, it does have green status
Shared Secret: I have given a shared secret
Client address: allowed 5 devices
DNS setting: One question I have is what DNS should I provide here? My internet router's DNS? or my internet routers's default gateway?
Routers: Do I need to provide any routes?

4- I have created a new test account on my Macbook Pro to be able to use it for the VPN through my iPhone
5- Now I go to my iPhone and setup VPN as follows:

Server: I provide my public IP address that I get from http://www.google.com
Account: I enter the user name of the account I just created in my Macbook Pro
RSA SecurID: is set to off
Password: I provide the password of the account I created on my Macbook Pro
Secret: I enter the shared secret key
Send All Trafic: is set to On

I have also forwarded the following ports from my internet router to my Macbook Pro's local IP address:

Protocol Port Range
TCP 1723 - 1723
UDP 500 - 500
UDP 1701 - 1701
UDP 4500 - 4500

When I set VPN on on my iPhone I get the following error: "The L2TP-VPN server did not respond..."

Here is my VPN service log:
#Start-Date: 2013-10-27 15:16:26 CET
#Fields: date time s-comment
2013-10-27 15:16:26 CET Loading plugin /System/Library/Extensions/L2TP.ppp
2013-10-27 15:16:26 CET Listening for connections...

And here is my System Log when I do a VPN from the iPhone:
Oct 27 15:41:09 macbookpro.lan racoon[3222]: >>>>> phase change status = Phase 1 started by us
Oct 27 15:41:09 macbookpro.lan racoon[3222]: IKE Packet: transmit success. (Responder, Main-Mode message 2).
Oct 27 15:41:09 macbookpro.lan racoon[3222]: IKE Packet: receive success. (Responder, Main-Mode message 3).
Oct 27 15:41:09 macbookpro.lan racoon[3222]: IKE Packet: transmit success. (Responder, Main-Mode message 4).
Oct 27 15:41:09 macbookpro.lan racoon[3222]: Connecting.
Oct 27 15:41:12 macbookpro.lan racoon[3222]: IKE Packet: transmit success. (Phase 1 Retransmit).
Oct 27 15:41:46 --- last message repeated 3 times ---
Oct 27 15:41:46 macbookpro.lan racoon[3222]: IKE Packet: transmit success. (Phase 1 Retransmit).
Oct 27 15:42:04 macbookpro.lan racoon[3222]: IKE Packet: transmit success. (Phase 1 Retransmit).
Oct 27 15:42:49 --- last message repeated 1 time ---
Oct 27 15:42:49 macbookpro.lan racoon[3222]: IKE Packet: transmit success. (Phase 1 Retransmit).
Oct 27 15:43:46 --- last message repeated 1 time ---
Oct 27 15:43:46 macbookpro.lan racoon[3222]: IKEv1 Phase 1: maximum retransmits. (Phase 1 Maximum Retransmits).
Oct 27 15:43:46 macbookpro.lan racoon[3222]: Phase 1 negotiation failed due to time up. dfec20fdf9615471:b34d569a7e265609

I would appreciate if you tell me what am I doing wrong?

mvmanolov · Oct 27, 2013

This is issues that some are experiencing - me included.
see:
https://forums.macrumors.com/showthread.php?p=18242103#post18242103

also there are a couple apple discussion forums where people are talking about this exact issue.

mohammad · Feb 1, 2014

I have now got the latest OS X Mavericks VPN fixes installed, but I'm still unable to VPN from my iPhone to my OS X server.. I'm now getting the following error in my OS X server VPN log... what am I doing wrong?

2014-02-01 23:52:04 CET Incoming call... Address given to client = 192.168.3.226
2014-02-01 23:52:04 CET --> Client with address = 192.168.3.226 has hung up

And this is a dml from TCPDUMP

00:05:38.957807 IP 84.xxx.xxx.242.37437 > macbookpro.lan.isakmp: isakmp: phase 1 I ident
00:05:38.958671 IP macbookpro.lan.isakmp > 84.xxx.xxx.242.37437: isakmp: phase 1 R ident
00:05:39.268785 IP 84.xxx.xxx.242.37437 > macbookpro.lan.isakmp: isakmp: phase 1 I ident
00:05:39.274870 IP macbookpro.lan.isakmp > 84.xxx.xxx.242.37437: isakmp: phase 1 R ident
00:05:39.369090 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: NONESP-encap: isakmp: phase 1 I ident[E]
00:05:39.369912 IP macbookpro.lan.ipsec-msft > 84.xxx.xxx.242.37444: NONESP-encap: isakmp: phase 1 R ident[E]
00:05:39.370310 IP macbookpro.lan.ipsec-msft > 84.xxx.xxx.242.37444: NONESP-encap: isakmp: phase 2/others R inf[E]
00:05:40.185232 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: NONESP-encap: isakmp: phase 2/others I oakley-quick[E]
00:05:40.186126 IP macbookpro.lan.ipsec-msft > 84.xxx.xxx.242.37444: NONESP-encap: isakmp: phase 2/others R oakley-quick[E]
00:05:40.348325 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: NONESP-encap: isakmp: phase 2/others I oakley-quick[E]
00:05:40.348329 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: UDP-encap: ESP(spi=0x0d3ef4d3,seq=0x1), length 132
00:05:41.414004 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: UDP-encap: ESP(spi=0x0d3ef4d3,seq=0x2), length 132
00:05:43.257222 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: UDP-encap: ESP(spi=0x0d3ef4d3,seq=0x3), length 132
00:05:47.250902 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: UDP-encap: ESP(spi=0x0d3ef4d3,seq=0x4), length 132
00:05:51.185552 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: UDP-encap: ESP(spi=0x0d3ef4d3,seq=0x5), length 132
00:05:55.545039 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: UDP-encap: ESP(spi=0x0d3ef4d3,seq=0x6), length 132
00:05:59.231882 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: UDP-encap: ESP(spi=0x0d3ef4d3,seq=0x7), length 132
00:05:59.846227 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: isakmp-nat-keep-alive
00:06:00.767580 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: NONESP-encap: isakmp: phase 2/others I inf[E]
00:06:00.767664 IP 84.xxx.xxx.242.37444 > macbookpro.lan.ipsec-msft: NONESP-encap: isakmp: phase 2/others I inf[E]

noremacyug · Apr 11, 2014

How To Properly Setup VPN Server? (Not Working)

Did this ever get resolved? I too am trying to setup the vpn on my mini running mavericks with osx server, latest updates installed. I've setup the vpn before on my router with no issues, but for the life of me I cant get my iphone to connect to my mini's vpn. I have forwarded the aforementioned ports and have a dyndns account. I have no issues connecting to other services on my mini from the wan. I could easily be doing something wrong, but I don't know what it would be, it looks pretty straight forward.

mvmanolov · Apr 11, 2014

noremacyug said:
Did this ever get resolved? I too am trying to setup the vpn on my mini running mavericks with osx server, latest updates installed. I've setup the vpn before on my router with no issues, but for the life of me I cant get my iphone to connect to my mini's vpn. I have forwarded the aforementioned ports and have a dyndns account. I have no issues connecting to other services on my mini from the wan. I could easily be doing something wrong, but I don't know what it would be, it looks pretty straight forward.

the latest server app does not have these issues, it was fixed a while back with 10.9.2

noremacyug · Apr 11, 2014

mvmanolov said:
the latest server app does not have these issues, it was fixed a while back with 10.9.2

I don't know what I'm doing wrong then.

satcomer · Apr 11, 2014

Then try the cheap VPN Server Agent and the free VPN Client Configurator.
You can seed video here:

noremacyug · Apr 12, 2014

thanks, but i'd like to try to make it work as is without any added apps. i even put in my ip address both server and client side. still no joy. the password and secret key match up, the ports are forwarded..... what gives? why won't this work? i almost have to be overlooking something. i really wish that apple had some better documentation on this. even though it looks simple, this has been a headache.

mus0r · Apr 13, 2014

Here's what you do.

Google search Asus (or whatever brand your prefer) routers with VPN built-in. Buy it. Enjoy a grief-free VPN.

I gave up on Apple's VPN service a long time ago.

noremacyug · Apr 14, 2014

Yeah. The vpn on my router works fine (asus rt-ac66u) It's more of a personal quest to get the vpn in my Mac working now.

noremacyug · May 5, 2014

Time to resurrect this.

So, just a few minutes ago I tried connecting to my home vpn from my MBA whilst away from home, worked perfectly. However, when plugging those same credentials into my iphone, it won't connect. What gives?

Alrescha · May 5, 2014

noremacyug said:
Time to resurrect this.

So, just a few minutes ago I tried connecting to my home vpn from my MBA whilst away from home, worked perfectly. However, when plugging those same credentials into my iphone, it won't connect. What gives?

Depending upon how your router deals with IPSec, "a few minutes" may not be long enough for your router to forget about your Macbook. It may only be able to deal with one IPSec tunnel at a time (it would not be the first).

A.

noremacyug · May 5, 2014

Alrescha said:
Depending upon how your router deals with IPSec, "a few minutes" may not be long enough for your router to forget about your Macbook. It may only be able to deal with one IPSec tunnel at a time (it would not be the first).

A.

I have an ipfire box built for routing/firewall. The vpn server is on my Mac mini. I simply cannot get my phone to connect. I can connect using my MBA tethered to my phone however, which I know isn't apples to apples. But at least I know att isn't blocking ports or whatnot. It's really annoying.

satcomer · May 6, 2014

I also came up with a video for 10.9.x Server VPN called Mavericks Server Part 16: VPN:

This could help in setting up VPN in OS X 10.9.x Server.

noremacyug · May 6, 2014

satcomer said:
I also came up with a video for 10.9.x Server VPN called Mavericks Server Part 16: VPN:

YouTube: video

This could help in setting up VPN in OS X 10.9.x Server.

Yeah, I've already seen his video. I have it all set like he shows and it will work with my MBA, just not my iphone. I don't get it.

satcomer · May 7, 2014

noremacyug said:
Yeah, I've already seen his video. I have it all set like he shows and it will work with my MBA, just not my iphone. I don't get it.

What service to you use for the domain? Did you go with No-IP? What do you use?

noremacyug · May 7, 2014

Using my dyndns account

mwb · May 7, 2014

Have you made sure Back to My Mac is turned off?

Edit: Never mind, I see you can already VPN from your MBA.

mohammad · May 11, 2014

satcomer said:
I also came up with a video for 10.9.x Server VPN called Mavericks Server Part 16: VPN:

YouTube: video

This could help in setting up VPN in OS X 10.9.x Server.

Thanks for the videos I looked at them and tried setting them as you have shown with .private but I still cannot connect with my iPhone to my MacbookPro which is running the server..

I always get the following error

2014-02-01 23:52:04 CET Incoming call... Address given to client = 192.168.3.226
2014-02-01 23:52:04 CET --> Client with address = 192.168.3.226 has hung up

What does this error actually mean?
What should I do next? :-(

AppleNinja88 · May 12, 2014

Things to check

Did you save a server VPN configuration profile and load it on your iPhone? This would probably help you with a lot of the problems your having on your iPhone. What kind of router do you have? You can control Apple Airport Routers with the Server app which open the correct VPN ports. If you can connect from your MBA then there is probably a setting your missing in your iPhone(referring back to the VPN configuration profile. Also, Where are you connecting from when your on your iPhone? Wifi or Cellular network? I would try a wifi connection first (other than your house duh... a good one too, not starbucks or some lame **** like that) Connecting over a cellular network is kinda spotty depending on service. I have also noticed when connecting over my iPhone's cellular network that you have to attempt the connection a couple times before it connects sometimes, are you trying this more than once or when it fails once or do you panic and then try to figure out if there is a problem. Do you have a static IP address from your ISP? or are you using a service that is updating the IP for a domain name automatically? You can find out what your outside ip address is by typing whats my ip in google, you most likely do not have a static IP. Hope some of this helps. Let me know how it goes buddy.

mohammad · May 18, 2014

AppleNinja88 said:
Did you save a server VPN configuration profile and load it on your iPhone? This would probably help you with a lot of the problems your having on your iPhone. What kind of router do you have? You can control Apple Airport Routers with the Server app which open the correct VPN ports. If you can connect from your MBA then there is probably a setting your missing in your iPhone(referring back to the VPN configuration profile. Also, Where are you connecting from when your on your iPhone? Wifi or Cellular network? I would try a wifi connection first (other than your house duh... a good one too, not starbucks or some lame **** like that) Connecting over a cellular network is kinda spotty depending on service. I have also noticed when connecting over my iPhone's cellular network that you have to attempt the connection a couple times before it connects sometimes, are you trying this more than once or when it fails once or do you panic and then try to figure out if there is a problem. Do you have a static IP address from your ISP? or are you using a service that is updating the IP for a domain name automatically? You can find out what your outside ip address is by typing whats my ip in google, you most likely do not have a static IP. Hope some of this helps. Let me know how it goes buddy.

Hi, I had never tried saving the VPN configuration profile and then loading that onto my iPhone but I still cannot connect and get the same message :-( I do not have a static IP but the dynamic IP that I get from my ISP does not change (as far as I know and have checked)

Nevertheless I started this thread and at the top I did explain how I have set things up, but appreciate if you can help me out on this. Below is the details:

1- Installed OS X server on my Mabckook Pro, I have tried both .lan and .private for using VPN (I haven't tried using a domain name yet)
2- They only services that was set to ON is VPN
3- Here are the settings for VPN:
Status is green and it says 'Available at macbookpro.private'
Configure VPN for: 'L2TP'
VPN Host Name: I have provided the IP address of my Macbook Pro that it gets form my local internet router, it does have green status
Shared Secret: I have given a shared secret
Client address: allowed 5 devices
DNS setting: One question I have is what DNS should I provide here? My internet router's DNS? or my internet routers's default gateway?
Routers: Do I need to provide any routes?
4- I have created a new test account on my Macbook Pro to be able to use it for the VPN through my iPhone
5- Now I go to my iPhone and setup VPN as follows:
Server: I provide my public IP address that I get from http://www.google.com
Account: I enter the user name of the account I just created in my Macbook Pro
RSA SecurID: is set to off
Password: I provide the password of the account I created on my Macbook Pro
Secret: I enter the shared secret key
Send All Trafic: is set to On

I have also forwarded the following ports from my internet router to my Macbook Pro's local IP address:

Protocol Port Range
TCP 1723 - 1723
UDP 500 - 500
UDP 1701 - 1701
UDP 4500 - 4500

When I use the VPN on on my iPhone I get the following error: "The L2TP-VPN server did not respond..." and in the VPN job I get the following '2014-05-18 21:12:34 CEST --> Client with address = 192.XXX.XXX.XX has hungup'

bilbo--baggins · May 24, 2014

I got it working

I followed the youtube video given in this thread, and at first I got the same message about the server not responding.

I changed 2 things

1) Instead of entering the settings on the iPhone, I emailed the Configuration Profile to my iPhone
2) After installing it on the iPhone, I then replaced the Shared Secret (using 1Password).

To answer your questions about DNS:
I specified 2 forwarding servers - which are OpenDNS addresses
My router has my servers IP address for DNS (the router is doing DHCP, so presumably it gives out my servers IP address to other devices on the network).
In terms of setting up DNS in Mavericks Server, I let it do the set up for me.

For reference (though I don't think it matters too much) my VPN hostname is server.imac.private and this gets put into the Configuration Profile, so on the iPhone I replaced this with my external IP address (found by googling What is my IP).

I have no routes configured.

I opened up the relevant ports on my router, which are the ones you listed.

Iamgort · Jun 12, 2014

http://code.macminivault.com/ got me mostly there, then I just had to fix a few things in what subnet I was providing DNS for IIRC.

Also this: http://blog.macminicolo.net/post/67570761408/setup-a-vpn-server-with-mavericks-server-10-9

How To Properly Setup VPN Server? (Not Working)

macrumors member

macrumors 6502a

macrumors member

macrumors member

macrumors 6502a

macrumors member

Suspended

macrumors member

macrumors regular

macrumors member

macrumors member

macrumors 68020

macrumors member

Suspended

macrumors member

Suspended

macrumors member

macrumors newbie

macrumors member

macrumors newbie

macrumors member

macrumors 6502a

macrumors regular

Our Staff