Networking Help

[appleguru1]

Ok, so I'm trying to set up a bit of a complicated network environment under Snow Leopard 10.6.4. I have an ethernet connection. That connection uses a Cisco IPSec VPN tunnel and a proxy server to connect to the internet. I then want to connect to a second VPN server outside the network (VPN in a VPN), and route all my traffic through that. I'm using Apple's built in VPN clients for both connections, but could try Cisco's VPN client for the initial connection if it will help.

If I use IPSec for the second VPN, it doesn't work at all. If I use PPTP, it connects fine. But (almost) all traffic still gets routed through the first VPN connection, despite the second one being higher in the service order. The only traffic that seems to make it to the second connection are requests for local IP addresses (192.168.1.*). Any tips? And yes, "Send all traffic through VPN connection" is checked too.

[belvdr]

What a mess. Unless you have a really good reason for doing so, I'd advise against it. It most likely will require a reconfig on either VPN server.

[appleguru1]

Yes, don't have much of an option, and I can't control the first VPN/change any settings (Though the second one is my own setup, so I can). The first VPN is required to get access out to the internet here.

[belvdr]

IPsec introduces new headers and I think the packets are getting fragmented. I've never heard of a VPN to get to an insecure network. That goes directly against what a VPN is for.

[appleguru1]

BTW, here's the relevant output from ifconfig (en0 is the ethernet connection, utun0 is the cisco IPSec VPN connection, and ppp0 is the PPTP VPN connection.. MAC Addresses are redacted):


en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
ether 00:[...]
inet6 fe80::[...] prefixlen 64 scopeid 0x4
inet 172.17.127.111 netmask 0xffffff00 broadcast 172.17.127.255
media: autoselect (100baseTX <full-duplex>)
status: active
utun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1280
inet 10.37.131.241 --> 10.37.131.241 netmask 0xffffffff
ppp0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1232
inet 192.168.1.59 --> 192.168.1.30 netmask 0xffffff00

[appleguru1]

Originally Posted by belvdr

IPsec introduces new headers and I think the packets are getting fragmented. I've never heard of a VPN to get to an insecure network. That goes directly against what a VPN is for.

I don't think they are though... I just don't think traffic is getting routed properly. When I have both networks connected, I can access my home router at 192.168.1.1 (This is on the second VPN), and everything works fine there.. I just can't access out to the web through that connection, requests to anything but 192.168.1.* go through the first VPN.

[appleguru1]

and indeed, this proves it... so I guess I just need to figure out how to properly modify the routing table, and why SL isn't routing like its supposed to based on service order:

$ route get 192.168.1.1
   route to: 192.168.1.1
destination: 192.168.1.0
       mask: 255.255.255.0
  interface: ppp0
      flags: <UP,DONE,STATIC,PRCLONING>
 recvpipe  sendpipe  ssthresh  rtt,msec    rttvar  hopcount      mtu     expire
       0         0         0         0         0         0      1232         0 
$ route get apple.com
   route to: quicktimestreaming.net
destination: quicktimestreaming.net
  interface: utun0
      flags: <UP,HOST,DONE,WASCLONED,PROTO3,IFSCOPE>
 recvpipe  sendpipe  ssthresh  rtt,msec    rttvar  hopcount      mtu     expire
       0         0         0         0         0         0      1280        20

Note the route to apple.com is over utun0, but the route to 192.168.1.1 is over ppp0...

[appleguru1]

...and here indeed is the problem.. with this setup, I have multiple default routes!

At the top of my routing table:

$ netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags        Refs      Use   Netif Expire
default            utun0              UCS            12        0   utun0
default            172.17.127.254     UGScI           1        0     en0
default            192.168.1.30       UGScI           0        0    ppp0

...and utun0 is higher than ppp0, despite the service order being set otherwise... can I change this?

[appleguru1]

...and there we go

sudo route delete default
sudo route add 192.168.1.30

That, (combined with disabling the proxy settings for the IPSec VPN), got me sorted... now data goes through the nested VPNs as it should!

Now the real question is why SL has a bug that doesn't respect service order for the default gateway...

BTW, deleting the default route just deleted the top, UGS flagged entry in the routing table. The UGScI routes remained. Anyone know what those flags mean? Adding the new default route added an entry flagged UGSc to the top.

[tunerX]

You have to modify the routes that get delivered by the Cisco VPN server or have your homegrown server send out a set of routes with a longer match.

accept the default route from the Cisco VPN server. Then have your private VPN server not send the default route but instead send more specific routes for destination networks that you want to be nested inside the VPN.

[tunerX]

If they are both cisco VPN servers then you would just need to modify the access-list associated with the isakmp client configuration group.

First tunnel - First VPN server
ip access-list extended ACL_OUTSIDE_TUNNEL
permit ip any any !(this generates the default route)

Nested tunnel - Second VPN server
ip access-list extended ACL_NESTED_TUNNEL !(Use this to route specific)
permit ip 192.168.0.0 0.0.255.255 any
permit ip 177.0.0.0 0.0.0.255 any
permit ip 66.179.0.0 0.0.255.255 any

[appleguru1]

Originally Posted by tunerX

You have to modify the routes that get delivered by the Cisco VPN server or have your homegrown server send out a set of routes with a longer match.

accept the default route from the Cisco VPN server. Then have your private VPN server not send the default route but instead send more specific routes for destination networks that you want to be nested inside the VPN.

Problem is I don't know what routes I want ahead of time...

So, when I connect to the second VPN server, I want *all* traffic to be routed through it. This should be accomplished via setting the service order in System Preferences, but the system seems to ignore the ordering and just sets the cisco VPN as the default route regardless of service order. My temporary 'fix' in post 9 fixes this routing, and makes the system perform as I desire it to.

And no, they're not both Cisco VPN servers; just the first one is. The second server is running on an old iMac G5 running leopard client back home. I have both PPTP and IPSec working to that machine, but only PPTP works inside an existing IPSec tunnel.

[appleguru1]

Bug report filed with apple for the incorrect service order, radar # 8232277

[belvdr]

What's the metric on those routes? Just because they are displayed in that order doesn't necessarily mean they are routed in that order.

[appleguru1]

Originally Posted by belvdr

What's the metric on those routes? Just because they are displayed in that order doesn't necessarily mean they are routed in that order.

Not sure; that's why I was asking what the flags meant.. It appears that only the top "UCS" flagged route is the default route that gets actually used (Or the "UGSc", but not the "UGScI routes"); if it gets removed the secondary "default" routes aren't used, at all. (In fact, trying to remove the default route after removing the top default route results in an error that the specified route doesn't exist, even though the other "defaults" are clearly still in the routing table).

...not sure how to display the metrics in os x... but reading netstat's manpage gives me this:

The routing table display indicates the available routes and their status.  Each route consists of a destination host or network and a gateway to use in
     forwarding packets.  The flags field shows a collection of information about the route stored as binary choices.  The individual flags are discussed in
     more detail in the route(8) and route(4) manual pages.  The mapping between letters and flags is:

     1       RTF_PROTO1       Protocol specific routing flag #1
     2       RTF_PROTO2       Protocol specific routing flag #2
     3       RTF_PROTO3       Protocol specific routing flag #3
     B       RTF_BLACKHOLE    Just discard packets (during updates)
     b       RTF_BROADCAST    The route represents a broadcast address
     C       RTF_CLONING      Generate new routes on use
     c       RTF_PRCLONING    Protocol-specified generate new routes on use
     D       RTF_DYNAMIC      Created dynamically (by redirect)
     G       RTF_GATEWAY      Destination requires forwarding by intermediary
     H       RTF_HOST         Host entry (net otherwise)
     I       RTF_IFSCOPE      Route is associated with an interface scope
     i       RTF_IFREF        Route is holding a reference to the interface
     L       RTF_LLINFO       Valid protocol to link address translation
     M       RTF_MODIFIED     Modified dynamically (by redirect)
     m       RTF_MULTICAST    The route represents a multicast address
     R       RTF_REJECT       Host or net unreachable
     S       RTF_STATIC       Manually added
     U       RTF_UP           Route usable
     W       RTF_WASCLONED    Route was generated as a result of cloning
     X       RTF_XRESOLVE     External daemon translates proto to link address

...which indicates that the "fake" default routes are routes that are associated with an interface scope ("I" flag).

[appleguru1]

Ok, I'm starting to think this is a configuration issue on my VPN server... Even when I connect to it directly on the iPhone, which it does just fine, traffic still doesn't get routed to it. I, as I did on my desktop machine, have to manually add the correct route before traffic will go over it. Any idea what settings I need to change on my server to get it behaving properly? I semi-confirmed this by using a different, properly behaving VPN server, and traffic gets routed correctly straight away.

[belvdr]

It sounds like the subnets aren't defined correctly on the VPN server. Sometimes these are defined by access lists (Cisco) or encryption domains (Check Point).

[appleguru1]

server is OS X (client)'s vpnd

[appleguru1]

...I have a feeling I'm going to need to manually edit the configuration plist and play with the offered keys, by adding something along these lines:

vpn:Servers:com.apple.ppp.pptp:IPv4:OfferedRouteAddresses:_array_index:0 = "192.168.1.0"
vpn:Servers:com.apple.ppp.pptp:IPv4:OfferedRouteTypes:_array_index:0 = "Public"
vpn:Servers:com.apple.ppp.pptp:IPv4:OfferedRouteMasks:_array_index:0 = "255.255.255.0"

...of course, I'd *love* to play with this, but doing so will very likely render my internet connection dead... so I might just have to wait a while to get it sorted

[belvdr]

The type seems weird. Can you simply change the type in the GUI from Public to Private? It should be labeled as Private so vpnd knows this is one of the internal subnets for clients to route to over the tunnel.

I doubt modifying the VPN config will hurt your Internet connection since vpnd isn't used for outbound access.