Tried improving DNS servers, now cannot access local IP addresses on LAN

[ChrisUnderChinkapins]

Hi, first post: I used namebench.app to discover Google's 8.8.8.8 DNS server appears fastest for me, so I added it to all the lists of DNS servers I could find, in Settings > Network and in AirportUtility for my TimeCapsule router and my two AirportExpress wireless access points. Now I can't go to my IP clock configuration page, or my IP security camera, both of which had fixed IP addresses on my LAN. The browser just times out.
I am trying to get rid of all the changes I've made but still no luck. And I have no idea what went wrong. One mistake I did not make was overwriting the range of IP addresses my router is not supposed to overwrite. However I did notice the first three parts of that IP address range are now 10.0.1 rather than 192.168.1 (IIRC) and I don't understand what that's about.
Any leads?
Thank you!

I have an iMac running High Sierra 10.13.2, and my TimeCapsule and AirportExpress devices got a firmware update about a week ago.

 

[techwarrior]

Are you accessing the clock and cameras by iP Address? Is you Mac getting an ip address in the same range as the clock and cameras?

In AP Utility > Network> Network Options, make sure the DHCP range is 192.168.1. 2 to whatever. The DHCP settings should be in the same rang as the static IP Addresses on your clock and cameras.

If you use hostnames to access the clock and camera, the DNS changes might be getting confused. Either reboot, or flush DNS (sudo dscacheutil -flushcache;sudo killall -HUP mDNSResponder).

On your Mac, if you are using DHCP, the only DNS server you should see is the Time Capsule address. Any DNS queries will start with the TC, if it can resolve the name locally, it will do so, then too it over to the Google DNS servers based on the Internet settings on the TC.

The other way you can use Google for your Mac, but allow other devices to use what your ISP provides is to leave the TC as DHCP for everything, and manually set the Mac DNS servers as Google, Google, Local. So, 8.8.8.8, 8.8.4.4, 192.168.1.1. You always want your router to be in the list of DNS servers if you resolve some hosts locally, since Google will have no idea about these.

 

[ChrisUnderChinkapins]

Thank you techwarrior! Right off the bat, you seem to have rescued me! Yes I was accessing them by IP address, and somehow my TimeCapsule was handing out 10.0... addresses rather than 192.168... addresses. I don't know why, and this seems to have changed when I made other changes. But I changed it to 192.168... and my Internet and local access is now working.
I do want to understand my system better, and make use of the faster 8.8.8.8, and I will study your suggestions further. But at least I am not *stuck* now, thank you for that!

 

[techwarrior]

Good to hear @ChrisUnderChinkapins that you are back in service.

Start by asking yourself what you hope to gain by using Google DNS. It might improve speeds to reach common sites, but is that really necessary on every device on your network?

Try the Mac first. Go into Network Preferences, click the Advanced button on whatever network device you use (Ethernet or WiFi). Leave the IPv4 set to DHCP (TCP\IP Tab), but add the Google DNS Servers to the list of DNS Servers on the left side of the DNS tab. Add 8.8.8.8, 8.8.4.4 and then your router (probably 192.168.1.1). Leave the search domain as is.

Use that for a week or so and see if you really notice any difference. If you think it is an improvement, go ahead and change the DNS on the router. Make sure DHCP doesn't change the local scope on the TC as you learned today.

Remove all of the DNS entries on your Mac and let the TC do the forwarding to the Google Servers. So, after changing the Mac back to default DNS, it should pick up the Time Capsule for DNS searches (192.168.1.1 should be the only entry in the DNS table).

So, DNS works as follows (using your Mac as example). You enter a hostname into whatever app you are using, Safari for example (say mail.comcast.net). It will first look at your /etc/hosts file to see if there is an address for the host, if not (usually not), it will then check local DNS cache to see if it already knows the address from prior attempts to connect to the host. If not in local cache, it will then use the first DNS server in the list (Your Router in most cases).

Similarly, each DNS server goes through a similar process, so it checks cache, then goes through its own list of DNS servers. If the first DNS lookup fails, it tries the second, then third, etc. At any of the above steps, if the IP Address is known, you proceed directly to the site and bypass all the subsequent steps.

If the only DNS server the Mac knows about is the TC Router, it leaves the rest of the search up to the router. Similarly, if Google doesn't know, it goes to the .com, .org, etc servers (who may only know the authoritative host for a given domain). If .net knows were dns.comcast.net is (but not mail.comcast.net), it queries dns.comast.net for mail.comcast.net

Since Google DNS servers have potentially far more cached content than your ISP (who may query Google if they cannot resolve the query), they are often faster than ISP servers. Basically, you are cutting out a middleman by taking extra steps on your router configuration.

You always want to use both of Google's (or OpenDNS) servers, this is in case one of the servers is not responding, the other will pick up the task. In reality, Google probably has several DNS servers that site behind each of the two IP Addresses to ensure load balancing and resiliency.

Sorry for the long post, but it is often useful to understand the concepts to understand why things improve by trying these things. Who knows, maybe it could make you the hit of the NYE party

 

[ChrisUnderChinkapins]

Techwarrior, this is really great of you. I appreciate this tremendously. I think I am much better equipped now -- but, I am curious, and hope you might humor me for some more questions. Not about exactly what I type where, but rather testing my understanding so far.

Here is how I understand you: Let's consider a computer (broadly meant to include desktops/laptops, appliances with routers in them, maybe my cable modem, servers at my ISP, and DNS servers elsewhere on the internet). When it has to handle an internet URL, it successively considers /etc/hosts, local cache, and consultations with servers in its list (or whichever of these things it has). If it still can't convert the URL to an IP address, it passes the URL up to whatever computer is above it. (or is "passing the URL up" the same step as "consultations with servers"?)

Does this mean that I actually don't have to put any DNS server addresses into any of my equipment anyplace? Is the only point of typing a DNS server address into one of my configuration panes simply to try to improve upon whatever would have happened further upstream -- for example, if people don't like how their ISP does the job?

As you say, I should ask myself what I hope to gain by using Google DNS. I understood DNS servers in a vague way and since I first got internet service faster than dialup -- a DirecWay satellite setup in, I think, 2003 -- whenever something had a place to specify them, I copied whatever I'd done before or whatever seemed to be default or already filled in elsewhere. Now I have internet service through the local cable TV company, and phone too (but not TV oddly enough). Here are the DNS server addresses I encountered in my current configurations when I took a closer look the other day:
10.0.1.1
198.77.116.8
198.77.116.12
24.154.1.34
24.154.1.37
24.154.1.38
It bothered me that I had little idea where I had originally found these. I now believe, for instance, the 198.77.116 ones are operated by DirecWay and offered in their docs, and yet I haven't used their service in years! So I wondered
what was a good way to choose DNS servers, and came across namebench.app and ran that. My understanding has changed (thank you), but it does still seem that I might enjoy some speed increase by using 8.8.8.8, if namebench.app tried thousands of tests of thousands of DNS servers and found it was fastest.

Perhaps a simpler and more minimalist approach would have been never to type a DNS server address into my equipment configurations anywhere.

I also read that users are typically fine just using the DNS servers their ISP uses. But I hunted around my cable modem and my ISP's web site, and could not find what those were. If I did know, would it have been redundant to type them in? That is, would I have wound up relying on them simply by not specifying any in my setup? Is that what somebody may have meant by "just using the DNS servers their ISP uses"?

Also, is there any point to specifying DNS servers in my multiple devices? Or would specifying them only in my Time Capsule do just as much good? They will all go through that on their way to internet locations. Perhaps if I do want to specify some servers, I only do it in the Time Capsule router.

You mention "You always want your router to be in the list of DNS servers if you resolve some hosts locally, since Google will have no idea about these." I understand I would want to grab requests for local host names before they get passed upstream out of my house, but, how would my router do that? my Time Capsule? I don't see where in its configuration options I could give it a list of local host names and LAN IP addresses. I should list them in the hosts file of my iMac, shouldn't I? Though of course that wouldn't work for any user here except me. Can my Time Capsule ask my iMac to resolve URLs? Perhaps I should give my iMac a fixed IP on my lan (it's been that way on and off for years anyway), and list my iMac's IP in the DNS servers pane for the Time Capsule??? I eagerly await learning whether this guess is brilliant or, instead, will get passed around internet forums for people to jeer and make sport.

I mentioned 8.8.8.8 and you said 8.8.4.4 in addition (this was in the spirit of "You always want to use both of Google's (or OpenDNS) servers...". Why 8.8.4.4 in particular? Did you happen to know it, or is there some basic principle to spelling out the IP address for the sidekick of any given DNS server?

For reference, my setup is:
Ethernet joins my iMac, my spouse's PC, the DS side of my Time Capsule, both of my AirPort Expresses, some network cameras, some network drives, our big TV, and perhaps other accessories, all through a 16 port switch in my basement rack of computer stuff shelves. Well, actually, the TV and one AirPort Express go to a little switch in a cabinet in the living room, and there is a single Ethernet cable run from that little switch to the big switch, to save me running more wire to the living room when we got the TV.
Also, there's Ethernet from the US side of my Time Capsule to the DS side of my cable modem. And telephone lines to my house wiring, as my cable modem is also the source of our "landline" phones (oddly enough we could not keep true landline phones working here as our buried phone cable would die as often as every few months, and the phone company kept replacing that 850' run).
Also, our iPhones and guests's phones use wireless, presumably to whichever of the 3 access points is strongest for them.

I am really enjoying understanding this just a bit better. I really appreciate you taking the time with me! Thank you!

 

[techwarrior]

Ok, so lets see how many of these I can capture...

First, Google "Google NS Servers" and you will find a number of articles talking about their two IP Addresses for their "Public DNS".

If you have an IP and want to know what it's hostname is, in Terminal, type "nslookup 8.8.8.8" and it will return the host name google-public-dns-a.google.com. By virtue of the domain "google.com" you know this host google-public-dns-a is within the google.com authoritative realm. So, for those others in your list, you can repeat to see what they are. But remember, the first DNS server that can resolve your query will stop the search, so adding more DNS servers won't help speed things up, they will probably never be used in practice.

You really only need the two google DNS servers. Most likely, these two addresses are addresses for a load balancer in two data centers, with several actual DNS servers behind these balancers. So, with the two addresses, if one data center is offline, the other should be online, and if one of the hosts is offline, the others can respond. The uptime for the Google DNS servers is probably as close to 100% as possible in the internet world, and may be one of the only real benefits of google DNS versus your ISP.

DNS is a hierarchical database that is implemented at various levels across the internet. The top level is .com, .net, .org, etc. At this level, any registered domain is recorded along with information you can sometimes get by typing whois google.com or whois comcast.net, etc (some domains choose to keep this info private). This tells you details about who registered the domain, how to contact them, etc. Included in the top level registration information is the authority for the domain, meaning the Name Server (DNS) that can resolve specific host names within the domain. For instance, google-public-dns-a is a host within the google domain. When you typed nslookup, your query went to the first DNS server in your chain that knew the answer. If none of your DNS servers knew, .com would have queried the authoritative server for the google.com domain to find that host. But, if any DNS server between you and .com knows this answer (because it recently did the lookup form someone else), it stops the chain and returns the answer to whoever requested it, who in turn responds to it's requestor and so on. DNS caches contain a Time To Live (TTL) that will drop the entry from cache over time to ensure the information is current. So, lower volume servers may not be cached anywhere and have to go all the way through the chain to resolve the hostname.

When your browser takes your input for a url, it parses the host + domain information from the URL (mail.comcast.net for example) and looks in /etc/hosts for the IP Address. If it doesn't find, it then looks in your local DNS cache (which would be there if you had previously tried to connect to this host). If no answer in cache, it then follows your resolver instructions (with *nix computers this is a file in /etc/ called resolv.conf but macOS actually uses a command scutil --dns to to get your resolver settings. From this, it knows to use the DNS server(s) in your network settings that were either statically configured or obtained from DHCP.

So, lets talk about DHCP for a minute. Dynamic Host Configuration Protocol (DHCP) is a simple way to configure hosts on a network. When a host starts up the network processes, it looks to see if an IP Address is statically assigned to the network interfaces that are configured to be active on your machine. In the case of a Mac or PC, this will typically be the WiFi and Ethernet interfaces. In the case of your router, it is the WAN Ethernet port. If there is no static address assigned, it sends a DHCP discovery broadcast out to the network and hopes there is a DCP server somewhere that will respond. In most cases, this is your home router, though in more complex networks, it might be a server such as a Windows 2016 Server with DHCP services, a macOS Server, a Linux or Unix server, etc. In the case of your router, it is the ISP network that responds.

A DHCP server will generally provide an IP Address to any host that asks for one (your ISP actually filters by MAC Address to ensure the Modem is registered as an authorized device). It also provides some additional information such as domain information, the router IP Address, and DNS servers. For IP phones, a lot of configuration info can be passed using DHCP, such as the server to register to, the server to get settings files from, etc.

Your Time Capsule simplifies a lot of this by responding with an IP Address for the requestor, along with the router IP Address (itself) and DNS server (itself). The TC also queries the Mac, PC or any device requesting an address to get it's host name and then caches it for future use. So, lets say your Mac is named "iMac" (in your sharing properties). The TC would create a DNS cache entry for iMac 192.168.1.10 for example. Now, if any host on your network tries to access iMac (for file sharing for example), the router would provide your IP Address to the host trying to make the connection. This is done dynamically, and stored in the router's DNS cache, so you never see it.

Your TC does something similar when it starts up. It will see no static IP (but maybe static DNS) and send a DHCP discovery to your ISP. Your ISP will return the domain, an IP Address for the WAN port on your router, and DNS servers. If you statically assigned DNS servers (Google), TC will ignore the DNS servers from your ISP. Your TC also has a static LAN IP Address (192.168.1.1 I assume) which is set when you specify the DHCP scope for your network (it picks .1 for itself and then provides addresses in the range you specify.

Ok, so back to the URL in your browser. So, your Mac didn't have any idea locally about the address for mail.comcast.net. So, it sees the TC as the only DNS server it is aware of (because of DHCP and no static DNS servers configured on it). So it sends a DNS query (actually "nslookup mail.comcast.net") to your TC. If the cache in your router knows this from another attempt to reach this host, it answers. If it doesn't know, it does an nslookup to it's DNS servers (either from the static assigned, or if you are using default, from the DHCP provided DNS servers). So, then your ISP does what your router did, look in cache, then send an nslookup to it's upstream DNS servers and so on. Again, at any place in the process, if an answer is known, it is returned and the chain reverses till it reaches your device. Your device then sends the http://ipaddress/whatever with the ip address it got from nslookup to fetch the website. It works similar for other network protocols like FTP, SSH, SCP, etc.

So, tying it all together, Since Google Name Servers are commonly used by a lot of users, including ISPs, they often know more hosts than your ISP does. But, if your ISP is large enough, the Google servers will be no faster than your ISP. By using Google's DNS servers on your router, or Mac, you are making the assumption that they will be faster, or more reliable than your ISP.

In reality, most of the large ISP DNS servers are probably going to do the job just fine. Google MAY be slightly faster because they are used more than the others, but if you run an nslookup with different DNS servers configured, you probably wouldn't notice the difference, it happens that fast. Which is to say, changing DNS servers won't make a slow ISP connection that much faster that you would notice. Where it does make a difference is with obscure sites that are not accessed as often. Since many ISP may use Google to resolve unknown hosts, Google MAY return results faster. Certainly, google will go direct to .com level where your ISP may go to google first.

By statically assigning Google DNS on your router, and letting DHCP tell your devices to ask the router for name resolution, you are simplifying configuration by doing it once on the router instead of manually on each device. Bypassing your router by manually configuring DNS on every device is both time consuming, and will only shave a couple of milliseconds off of queries so, more hassle than it is worth probably.

 

[HenryAZ]

I would point your router, or edge device, to a reliable name server, such as Google or your ISP. Then point every other device on your LAN, either via DHCP or static configuration, to your router for name service. This has the great advantage of maintaining a cache on your local network (i.e. in your edge router). Anything not in the local cache will be forwarded by your edge router to the outside name server, which also has a good cache. Layer on layer. Keep a good local cache available to your LAN, and the let your edge device discover what it does not know.

 

[techwarrior]

I would point your router, or edge device, to a reliable name server, such as Google or your ISP. Then point every other device on your LAN, either via DHCP or static configuration, to your router for name service. This has the great advantage of maintaining a cache on your local network (i.e. in your edge router). Anything not in the local cache will be forwarded by your edge router to the outside name server, which also has a good cache. Layer on layer. Keep a good local cache available to your LAN, and the let your edge device discover what it does not know.

The best reason to config external DNS servers on your router, and point LAN hosts to the router is for the same reasons for using DHCP, it is simply much easier and consistent to automatically configure network settings centrally rather than on each device.

DSN records have a Time To Live (TTL) which is typically 24 hours.When the TTL expires, the name server will re-query the next time a request is received. On most home LANs, the traffic is not so great that this will make any real difference when you consider the speed a DNS query takes. To illustrate, on Mac Terminal, type nslookup FQDN (nslookup www.yahoo.com for example) for a site you have not visited in weeks or months. Better yet, choose a site on a domain that gets very little traffic, to force it to go from your Mac <> Router <> Outside DNS <> Top Level (.com) DNS <> the domain's Authoritative Name Server.

Before you do it, reboot your Mac and router to clear DNS cache. You generally won't be able to count to 1 before the response comes back.

So, your router's DNS cache is not typically a significant factor in the speed of loading a website. The download speed of content, particularly sites with a lot of content or ads, and the name resolving for the linked content (ads primarily) will have a much greater impact on load speeds. You may have noticed web pages with a lot of ads take longer to finish loading than simple, static pages. It has to do with the back and forth of downloading content from a number of sites, and many DNS queries that are involved in opening complex web sites. Visiting a web site with external domain content links has to 1 - lookup the DNS for the primary site, 2 - send the http get request, 3 - download the html content, 4 - render the content, 5 - repeat all of the above for each link to content on other hosts. So, more eternally linked content requires the browser to get content from a lot of different web servers across the internet, often on many different domains.

Your Mac and router's cache helps offload some of the load on your ISP or external DNS servers, who in turn reduce the load on the top level DNS servers, and in turn on the target domain's authoritative name server. All of this was created in the days of very slow dial up modem connections, and was necessary to make even simple web pages load in a reasonable amount of time. With faster internet service, web sites could get more complex as the network speeds compensated for the extra work.

 

[HenryAZ]

The best reason to config external DNS servers on your router, and point LAN hosts to the router is for the same reasons for using DHCP, it is simply much easier and consistent to automatically configure network settings centrally rather than on each device.

DSN records have a Time To Live (TTL) which is typically 24 hours.

Most CDN providers have a TTL a low as 5 minutes. 24 hours my have been the standard a long time ago. Back when I ran a network, I set TTL to 7 days, as long as I was sure it would not change. The longer the TTL, the better your DNS performance.

 

[techwarrior]

Most CDN providers have a TTL a low as 5 minutes. 24 hours my have been the standard a long time ago. Back when I ran a network, I set TTL to 7 days, as long as I was sure it would not change. The longer the TTL, the better your DNS performance.

Static services like email that don't change often may use longer TTL. Dynamic services, or servers that are being prepared to move might use shorter TTL to avoid service disruptions. I suspect the 12-24 hours may still be pretty widely used.

I understand many ISP setup their DNS servers to ignore TTL and refresh their caches in off hours or some sort of schedule.