On 05/16/2012 10:11 AM, JD wrote: > I have nscd running. > /etc/resolv.conf starts out with > nameserver 127.0.0.1 > nameserver 192.168.1.254 > > > The 192.168.1.254 is the router, which has been a fast and reliable resolver. > > So, to test nscd caching behavior, > I browse (using FF) over to any website. > After some time, the address is resolved and the page comes up. > I kill the tab of the page, and open a new tab and aim the browser > at same url. Browser again says: looking up whatever....com and takes > several seconds to resolve it. > > I thought that nscd is supposed to cache the translation from the > first lookup. > > Am I to believe that the browser is NOT using /etc/resolv.conf? > If not, what is it using? > Or could it be that nscd is useless in this respect? > I've not looked at nscd in a long time....but I never could see the value in it and never could get it to what I thought was a working or useful configuration for my needs. No browser or application uses resolv.conf directly.  They make calls to the resolver libraries which in turn use it. IMO, if your router does caching name services there really is no benefit to having systems do their own caching since the overhead of local requests should be small. However, it seems that your router may not be caching since it is taking several seconds. In cases where the router isn't doing caching, or is doing it poorly, I prefer to simply run bind on a single server and point all the systems to it for resolution. With the current Fedora systems this is easy.  All one need to do is install bind and bind-chroot and enable/start the service.  On the "bind" host all you need is 127.0.0.1 defined as a nameserver.  Then, if you use a tool such as "wireshark" you will see that requests will only go out if the answer is not in the cache or the TTL has expired.

On 05/16/2012 02:54 PM, JD wrote: > I understand the libs are what make calls to the resolver. But even > the resolver must look > at /etc/resolv.conf. Well, you did say: "Am I to believe that the browser is NOT using /etc/resolv.conf" which to me reads that you were thinking that somehow the browser itself should be using resolv.conf. I'm sorry if I misread what you wrote. > If it is empty, NOTHING gets resolved. Not "entirely" true. With named not running..... [egreshko@f16-1 ~]$ cat /etc/resolv.conf # Generated by NetworkManager #search greshko.com #nameserver 192.168.0.55 [egreshko@f16-1 ~]$ ping misty PING misty (192.168.0.55) 56(84) bytes of data. 64 bytes from misty (192.168.0.55): icmp_req=1 ttl=64 time=1.99 ms since /etc/nsswitch.conf contains hosts: files dns and /etc/hosts contains 192.168.0.55 misty if you take the "files" out of the hosts line....then NOTHING gets resolved.

> I was using nscd thinking it is a lightweight caching resolver. But as > it turns out it is useless. > Time for fedora to bury it :) > Re: My router: it does very little if any caching - and has no > configuration for it at all. > > I will try bind. I've not used it....but have heard good things about dnsmasq which, according to yum info, is A lightweight DHCP/caching DNS server.

On 05/16/2012 04:27 PM, JD wrote: > I have used dnsmasq as well, and communicated with it's author > a couple of years ago. > But as I recall, it did not seem to help much, as I observed that > browsing to a website with just a couple of minutes lapse time, > FF showed on the status line it was looking up the domain. > Perhaps the cache expiry was set to a very short time? > > 2 things may be coming into play..... 1st it "could" be that there is a cache time setting in dnsmasq. 2nd the owners of a domain, and by definition, who control the settings of the master DNS server for their domains set up a TTL for their records. If they set this to a low value you may see this problem. As a mater of fact, I've seen cases where the TTL has been set to as low as 300 seconds.

when you do a look up on www.cnn.com it will return 4 IP addresses. Now, since bind would have that in its cache it wouldn't have to send out a query. What I don't know is if an application would make a request would the list be returned in the same order every time to the requesting application? In other words, if the TTL is not set low, would that defeat the round robin technique.

Interesting things to investigate.....if I really had the time.

On 05/17/2012 12:28 AM, Greg Woods wrote: > On Thu, 2012-05-17 at 00:13 +0800, Ed Greshko wrote: >> when you do a look up on www.cnn.com it will return 4 IP >> addresses. Now, since bind would have that in its cache it wouldn't have to send out >> a query. What I don't know is if an application would make a request would the list >> be returned in the same order every time to the requesting application? In other >> words, if the TTL is not set low, would that defeat the round robin technique. > Unfortunately, there is no guarantee that a client resolver will > actually use the IP addresses in the order they are presented by the DNS > server. Nothing in the DNS spec requires them to do so. Yeah.... I know there is no spec... I'm just expecting the clients to be "dumb" and take the first one in the stack. :-) :-) >> Interesting things to investigate.....if I really had the time. > My experience says that DNS round robining is actually a poor method of > load balancing. I'm surprised to see a large site like CNN resorting to > this (if that's really what they are doing this for). Perhaps in > combination with a low TTL and a modified DNS server, they can send out > a completely different set of IPs every few minutes, and achieve a sort > of crude load balancing that way, but I think load balancing works > better if you just send out a single IP and use a load balancer that you > can control, such as LVS (Linux Virtual Server) that can farm out > incoming connections to a single virtual address out to multiple real > addresses. Yes... I suppose one also has to ask if the load balancing is meant to be server or network balancing.

On 05/17/2012 03:52 AM, JD wrote: > Well, after running dnsmasq with the configuration I just emailed, > I see the following behavior of firefox vs. running nslookup on command line. > > FF, even after resolving google.com only a minute ago, is still spinning saying: > lookup up www.google.com > whereas , on the command line, I run nslookup www.google.com > and almost instantly, I get > Server: 127.0.0.1 > Address: 127.0.0.1#53 > > Non-authoritative answer: > Name: google.com > Address: 74.125.239.1 > Name: google.com > Address: 74.125.239.2 > Name: google.com > Address: 74.125.239.3 > Name: google.com > Address: 74.125.239.4 > Name: google.com > Address: 74.125.239.5 > Name: google.com > Address: 74.125.239.6 > Name: google.com > Address: 74.125.239.7 > Name: google.com > Address: 74.125.239.8 > Name: google.com > Address: 74.125.239.9 > Name: google.com > Address: 74.125.239.14 > Name: google.com > Address: 74.125.239.0 > > and FF is still spinning waiting for the resolution. > > Does anyone see this discrepancy? No.... Have you tried a different browser? Chrome, or konqueror for example? One SWAG would be to set network.dns.disableIPv6 to true in about:config.

On 05/17/2012 02:06 PM, JD wrote: > That value is already set to false. I'm suggesting you set it to "true" to disable IPv6. > Yes I did try Chrome. > Chrome resolves domain names as fast as nslookup . > After I browsed to a domain using Chrome, and it almost immediately > resolved and it brought in the page, > I quickly clicked on the FF workspace and browsed to same domain. > FF went into the looking up ...... spin dance. > Could it be that FF is not using same library as nslookup and Chrome? I'd be surprised if they used anything other than the standard gethostbyname()

On 05/16/2012 11:47 PM, JD wrote: > Interesting point. Hard to believe that  most domain controllers would that. > Must be a small  percentage? I have no idea what percentage it may be.  But 300 seems a bit low.... I'd have to do some research but it very well may be that sites set their TTL's low in an attempt to balance DNS server load with load balancing (round robin) of their web sites. It seems that the TTL for www.cnn.com is set to 150 seconds. I'd have to check....  But when you do a look up on www.cnn.com it will return 4 IP addresses.  Now, since bind would have that in its cache it wouldn't have to send out a query.  What I don't know is if an application would make a request would the list be returned in the same order every time to the requesting application?   In other words, if the TTL is not set low, would that defeat the round robin technique. Interesting things to investigate.....if I really had the time. -- Never be afraid to laugh at yourself, after all, you could be missing out on the joke of the century. -- Dame Edna Everage -- users mailing list users(a)lists.fedoraproject.org To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org

On 05/16/2012 04:27 PM, JD wrote: > I have used dnsmasq as well, and communicated with it's author > a couple of years ago. > But as I recall, it did not seem to help much, as I observed that > browsing to a website with just a couple of minutes lapse time, > FF showed on the status line it was looking up the domain. > Perhaps the cache expiry was set to a very short time? > > 2 things may be coming into play..... 1st it "could" be that there is a cache time setting in dnsmasq. 2nd the owners of a domain, and by definition, who control the settings of the master DNS server for their domains set up a TTL for their records. If they set this to a low value you may see this problem. As a mater of fact, I've seen cases where the TTL has been set to as low as 300 seconds.

On Tue, May 15, 2012 at 9:20 PM, Ed Greshko<Ed.Greshko(a)greshko.com> wrote: > On 05/16/2012 10:11 AM, JD wrote: >> I have nscd running. >> /etc/resolv.conf starts out with >> nameserver 127.0.0.1 >> nameserver 192.168.1.254 >> >> >> The 192.168.1.254 is the router, which has been a fast and reliable resolver. >> >> So, to test nscd caching behavior, >> I browse (using FF) over to any website. >> After some time, the address is resolved and the page comes up. >> I kill the tab of the page, and open a new tab and aim the browser >> at same url. Browser again says: looking up whatever....com and takes >> several seconds to resolve it. >> >> I thought that nscd is supposed to cache the translation from the >> first lookup. >> >> Am I to believe that the browser is NOT using /etc/resolv.conf? >> If not, what is it using? >> Or could it be that nscd is useless in this respect? >> > > I've not looked at nscd in a long time....but I never could see the value in it and > never could get it to what I thought was a working or useful configuration for my needs. > > No browser or application uses resolv.conf directly. They make calls to the resolver > libraries which in turn use it. > > IMO, if your router does caching name services there really is no benefit to having > systems do their own caching since the overhead of local requests should be small. > However, it seems that your router may not be caching since it is taking several seconds. > > In cases where the router isn't doing caching, or is doing it poorly, I prefer to > simply run bind on a single server and point all the systems to it for resolution. > > With the current Fedora systems this is easy. All one need to do is install bind and > bind-chroot and enable/start the service. On the "bind" host all you need is > 127.0.0.1 defined as a nameserver. Then, if you use a tool such as "wireshark" you > will see that requests will only go out if the answer is not in the cache or the TTL > has expired. > I understand the libs are what make calls to the resolver. But even the resolver must look at /etc/resolv.conf. If it is empty, NOTHING gets resolved. I was using nscd thinking it is a lightweight caching resolver. But as it turns out it is useless. Time for fedora to bury it :) Re: My router: it does very little if any caching - and has no configuration for it at all. I will try bind. Thanx Ed. JD

fedora skrev 16.05.12 10:33: > ... or try dnsmasq > > suomi > > On 05/16/2012 08:54 AM, JD wrote: >> On Tue, May 15, 2012 at 9:20 PM, Ed Greshko<Ed.Greshko(a)greshko.com> >> wrote: >>> On 05/16/2012 10:11 AM, JD wrote: >>>> I have nscd running. >>>> /etc/resolv.conf starts out with >>>> nameserver 127.0.0.1 >>>> nameserver 192.168.1.254 >>>> >>>> >>>> The 192.168.1.254 is the router, which has been a fast and reliable >>>> resolver. >>>> >>>> So, to test nscd caching behavior, >>>> I browse (using FF) over to any website. >>>> After some time, the address is resolved and the page comes up. >>>> I kill the tab of the page, and open a new tab and aim the browser >>>> at same url. Browser again says: looking up whatever....com and takes >>>> several seconds to resolve it. >>>> >>>> I thought that nscd is supposed to cache the translation from the >>>> first lookup. >>>> >>>> Am I to believe that the browser is NOT using /etc/resolv.conf? >>>> If not, what is it using? >>>> Or could it be that nscd is useless in this respect? >>>> >>> I've not looked at nscd in a long time....but I never could see the >>> value in it and >>> never could get it to what I thought was a working or useful >>> configuration for my needs. >>> >>> No browser or application uses resolv.conf directly. They make >>> calls to the resolver >>> libraries which in turn use it. >>> >>> IMO, if your router does caching name services there really is no >>> benefit to having >>> systems do their own caching since the overhead of local requests >>> should be small. >>> However, it seems that your router may not be caching since it is >>> taking several seconds. >>> >>> In cases where the router isn't doing caching, or is doing it >>> poorly, I prefer to >>> simply run bind on a single server and point all the systems to it >>> for resolution. >>> >>> With the current Fedora systems this is easy. All one need to do is >>> install bind and >>> bind-chroot and enable/start the service. On the "bind" host all >>> you need is >>> 127.0.0.1 defined as a nameserver. Then, if you use a tool such as >>> "wireshark" you >>> will see that requests will only go out if the answer is not in the >>> cache or the TTL >>> has expired. >>> >> I understand the libs are what make calls to the resolver. But even >> the resolver must look >> at /etc/resolv.conf. If it is empty, NOTHING gets resolved. >> I was using nscd thinking it is a lightweight caching resolver. But as >> it turns out it is useless. >> Time for fedora to bury it :) >> Re: My router: it does very little if any caching - and has no >> configuration for it at all. >> >> I will try bind. >> >> Thanx Ed. >> >> JD HI Why do you have 127.0.0.1 in /etc/resolv.conf? Could it be that your computer ask himself to resolv this ip and as he can't do that then he get to your router and ask? Do you have the same behaviour when only your router's ip adress is in /etc/resolv.conf? kind regards Daniel

On Tue, 15 May 2012 23:54:03 -0700 JD wrote: > I was using nscd thinking it is a lightweight caching resolver. But as > it turns out it is useless. > Time for fedora to bury it :) It isn't useless for me at work: It is the only thing that makes NIS lookups reliable. At some point in time, glibc apparently changed the timeout for NIS to something like 3 nanoseconds :-).

On Wed, 16 May 2012 09:42:20 -0600 JD wrote: >> It isn't useless for me at work: It is the only thing that makes >> NIS lookups reliable. At some point in time, glibc apparently >> changed the timeout for NIS to something like 3 nanoseconds :-). > 3 ns?? So, what did you do to make it work? The smiley was for the 3 ns value. I have no idea what it actually is, but NIS only functions error free if I am running nscd on my local system. The slightest amount of network traffic that slows down the local network always results in NIS errors unless I'm running nscd. This happened somewhere around fedora 12 or 13 I seem to recall. Never ran nscd before that, and never had a problem, and none of the local network infrastructure changed, so I figured it must be something in libc that was more sensitive to slight delays in responses.

On 16 May 2012 07:41, JD&lt;jd1008(a)gmail.com&gt; wrote: > I have nscd running. > /etc/resolv.conf starts out with > nameserver 127.0.0.1 > nameserver 192.168.1.254 > > > The 192.168.1.254 is the router, which has been a fast and reliable > resolver. > > So, to test nscd caching behavior, > I browse (using FF) over to any website. > After some time, the address is resolved and the page comes up. > I kill the tab of the page, and open a new tab and aim the browser > at same url. Browser again says: looking up whatever....com and takes > several seconds to resolve it. > > I thought that nscd is supposed to cache the translation from the > first lookup. > > Am I to believe that the browser is NOT using /etc/resolv.conf? > If not, what is it using? > Or could it be that nscd is useless in this respect? nscd should cache host lookups if it is configured correctly. Run nscd -g to see caching statistics and see if they tally with your usage. If all of them are misses then you need to run nscd in debug mode with 'nscd -d' to see why the caching is not working. Make sure you increase the value of debug-level to get verbose messages. An easy test is to use 'getent hosts<hostname>' and 'getent ahosts <hostname>' and study the nscd logs to see if you get a cache miss everytime.

On 05/15/2012 07:11 PM, JD wrote: > > I have nscd running. > /etc/resolv.conf starts out with > nameserver 127.0.0.1 If you're actually running a local caching name server (bind or dnsmasq), you don't need nscd.  Running both is overkill.  You're going to waste memory by having everything cached in two different places. You can't test nscd with nslookup, dig, or host.  All of those are DNS tools.  They'll use the contents of resolv.conf, but they do not use the libc host lookup functions, and therefore they will not use nscd.  If you want to test nscd, you should use "getent host".

On 05/17/2012 09:19 PM, JD wrote: > That's excellent info. contradicts what other people have replied. > I also meant to point out that if you select nscd rather than a local caching server, you don't need 127.0.0.1 in resolv.conf. glibc connects to nscd via a Unix socket rather than via IP. The same is true of sssd on newer releases of Fedora. (nscd is, I believe, deprecated by Fedora in favor of sssd). --

There should not be a configuration for that. If there is, then dnsmasq would be going against the recommendations of the DNS RFCs. The response to a DNS request includes a TTL (Time To Live). According to the RFC.... TTL which is the time to live of the RR. This field is a 32 bit integer in units of seconds, an is primarily used by resolvers when they cache RRs. The TTL describes how long a RR can be cached before it should be discarded. So, dnsmasq is dropping the records from its cache according to when the owner of the record wants it. This is how the DNS system is supposed to work.

So, what's to prevent someone from simply modifying dnsmasq (or any other open source caching name resolver) to change the expiration time to a value greater than what the owner of the domain wants? Sure it may result in using stale ip addresses once in a while. I think that's more tolerable than having to wait anywhere from 10 to 30 seconds to resolve every new name browsed to; (new relative to contents of the cache).

I am indeed pointing my resolv.conf to the 2 google nameservers. You're probably right about our home network. I think the router has a very low bandwidth (hardware wise), probably because it doubles up as the decoder for the TV contents being viewed on 2 to 3 different TV's in the house. TV signals come directly to the router on the coax, and then are sent back on the coax to the 3 TV sets. We're stuck with what we have.

That value is good for comparison only as it is not the real time. Take a look at this: time dig @8.8.8.8 www.ny.com ; <<>> DiG 9.8.2-RedHat-9.8.2-1.fc16 <<>> @8.8.8.8 www.ny.com ; (1 server found) ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24466 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;www.ny.com. IN A ;; ANSWER SECTION: www.ny.com. 3383 IN CNAME web1.ny.com. web1.ny.com. 3383 IN A 184.106.212.108 ;; Query time: 61 msec ;; SERVER: 8.8.8.8#53(8.8.8.8) ;; WHEN: Fri May 18 03:17:11 2012 ;; MSG SIZE rcvd: 63 real 0m10.07s user 0m0.01s sys 0m0.01s So, the router or my super lightly loaded laptop, or the ISP, must be the cause of such delays. $ w 03:20:45 up 5:52, 3 users, load average: 0.02, 0.02, 0.05

On 05/18/2012 05:40 PM, Ed Greshko wrote: > I can't see how your router or ISP could add to the "real" time. I suppose you don't > have another system to check it.... Could you try the non-builtin time command? /usr/bin/time dig @8.8.8.8 www.ny.com

On 05/18/2012 05:21 PM, JD wrote: > That value is good for comparison only as it is not > the real time. Take a look at this: > > time dig @8.8.8.8 www.ny.com > > ;<<>> DiG 9.8.2-RedHat-9.8.2-1.fc16<<>> @8.8.8.8 www.ny.com > ; (1 server found) > ;; global options: +cmd > ;; Got answer: > ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24466 > ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0 > > ;; QUESTION SECTION: > ;www.ny.com. IN A > > ;; ANSWER SECTION: > www.ny.com. 3383 IN CNAME web1.ny.com. > web1.ny.com. 3383 IN A 184.106.212.108 > > ;; Query time: 61 msec > ;; SERVER: 8.8.8.8#53(8.8.8.8) > ;; WHEN: Fri May 18 03:17:11 2012 > ;; MSG SIZE rcvd: 63 > > > real 0m10.07s > user 0m0.01s > sys 0m0.01s > > So, the router or my super lightly loaded laptop, > or the ISP, must be the cause of such delays. > > $ w > 03:20:45 up 5:52, 3 users, load average: 0.02, 0.02, 0.05 Something must be really strange on your system..... The same time dig command on my system(s) yield.... ;; Query time: 21 msec real 0m0.029s user 0m0.002s sys 0m0.005s Your query time is 61 msec which sounds reasonable. So, dig sent out and got back the response quite fast...and user and sys times seem reasonable...without knowing your hardware. I can't see how your router or ISP could add to the "real" time. I suppose you don't have another system to check it....

On 05/18/2012 12:22 AM, JD wrote: > I think that's more tolerable than > having to wait anywhere from 10 to 30 seconds to resolve every > new name browsed to; (new relative to contents of the cache). If the name isn't in your cache, you're going to have to look it up and using a longer TTL isn't going to change the lookup time one little bit. What it will do is keep recent addresses in the cache longer so that you won't have to look them up so often.

On 05/18/2012 10:35 AM, JD wrote: > On 05/18/2012 10:22 AM, Joe Zeff wrote: >> On 05/18/2012 12:22 AM, JD wrote: >>> I think that's more tolerable than >>> having to wait anywhere from 10 to 30 seconds to resolve every >>> new name browsed to; (new relative to contents of the cache). >> >> If the name isn't in your cache, you're going to have to look it up >> and using a longer TTL isn't going to change the lookup time one >> little bit. What it will do is keep recent addresses in the cache >> longer so that you won't have to look them up so often. > Not so. > If I could make them live in my local cache a little longer, > lookup will be in ms rather than seconds. Read what I wrote again, more carefully. If the site's new, and NOT IN YOUR CACHE, the TTL doesn't matter. This only means that sites you've been to recently stay in the cache longer, which isn't the same thing. Yes, I agree that they should hang around long enough to be useful but what I was pointing out is that it's not going to help on a NEW site, as you were saying.