« Interview with Shifd Creators: Nick Bilton and Michael Young, Pt. II | Main | High Performance Web Sites: Rule 10 – Minify JavaScript »
July 20, 2007
The Domain Name System (DNS) maps hostnames to IP addresses, just as phonebooks map people's names to their phone numbers. When you type www.yahoo.com into your browser, a DNS resolver contacted by the browser returns that server’s IP address. DNS has a cost. It typically takes 20-120 milliseconds for DNS to lookup the IP address for a given hostname. The browser can’t download anything from this hostname until the DNS lookup is completed.
DNS lookups are cached for better performance. This caching can occur on a special caching server, maintained by the user's ISP or local area network, but there is also caching that occurs on the individual user's computer. The DNS information remains in the operating system's DNS cache (the "DNS Client service" on Microsoft Windows). Most browsers have their own caches, separate from the operating system's cache. As long as the browser keeps a DNS record in its own cache, it doesn't bother the operating system with a request for the record.
Internet Explorer caches DNS lookups for 30 minutes by default, as specified by the DnsCacheTimeout registry setting. Firefox caches DNS lookups for 1 minute, controlled by the network.dnsCacheExpiration configuration setting. (Fasterfox changes this to 1 hour.)
When the client’s DNS cache is empty (for both the browser and the operating system), the number of DNS lookups is equal to the number of unique hostnames in the web page. This includes the hostnames used in the page’s URL, images, script files, stylesheets, Flash objects, etc. Reducing the number of unique hostnames reduces the number of DNS lookups.
Reducing the number of unique hostnames has the potential to reduce the amount of parallel downloading that takes place in the page. Avoiding DNS lookups cuts response times, but reducing parallel downloads may increase response times. My guideline is to split these components across at least two but no more than four hostnames. This results in a good compromise between reducing DNS lookups and allowing a high degree of parallel downloads.
Steve Souders
[Steve Souders is Yahoo!'s Chief Performance Yahoo!. This is one in a series of Best Practices for Speeding Up Your Web Site. This article is based on Steve's book High Performance Web Sites, published by O'Reilly.]
Posted at July 20, 2007 10:42 AM
Conceptually this conflicts with implementing your own CDN. It would be quite easy using Apache to include rewrite rules for imgsrv[0-9]{2}.myhost.com to map to your webserver so you can have 100 image *servers* that can be used for concurrent requests. But if these DNS hits are actually up to 100ms does this mean you can only use CDN's that are based on IP addresses?
Posted by: Nathan at July 25, 2007 11:59 AM
Yes, there's a tradeoff between reducing DNS lookups and maximizing parallel downloads. Maximizing parallel downloads is more important. Really, the DNS lookups you want to eliminate are the ones that are used for only a small number of requests. We've published a paper on this topic: http://yuiblog.com/blog/2007/04/11/performance-research-part-4/
Posted by: Steve Souders at July 28, 2007 11:17 AM
I'm optimizing an advertising server, don't have the opportunity for parallel downloads because each request only offers two addresses anyway (one on the image for audit tracking and the other for the creative.)
What services are available to move DNS request to the visitor edge? Who provides 'anycast', geographic round-robin, or distributed resolution?
Best,
Justin
Posted by: Justin Hitt at September 9, 2007 9:22 PM
Justin - AkaDNS (http://www.akamai.com/html/technology/products/enhanced_dns.html) is a popular anycast service from Akamai. There are many problems from ads that are solved by following these performance rules. In this case, if there are only two resources it would be better to serve both from the same domain. This might be more work on the Advertiser's side, to extract the audit tracking from the actual creative requests, but it'll cut the DNS lookups on the user's side by half.
Posted by: Steve Souders at September 9, 2007 10:04 PM
Steve,
Thanks. And to handle volume and reduce load on adservers ... place img[0-n].ads.DOMAIN.TLD behind a load balancer with ads.DOMAIN.TLD mapping ads.DOMAIN.TLD/img/ as the pool of creative. I'll still need to investigate other providers, AkaDNS is out of budget for many clients.
Best,
Justin
Posted by: Justin Hitt at September 17, 2007 2:13 PM
I`m wondering if yahoo has tested the effects of using the "base" tag in the header, and if improves the performance for DNS lookups.
It would be interesting to see.
Example:..
...
, Marc
Posted by: Marc at September 26, 2007 1:09 PM
We haven't tested that. It would be an interesting experiment. My hunch is it would have little effect on the overall response time of the page, but might slightly delay progressive rendering making for a worse user experience. The reason is that the DNS lookup would happen right away, and delay any rendering the browser was trying to do. It would be better to do DNS lookups "on demand", and allow non-blocked elements to appear as quickly as possible.
Posted by: Steve Souders at September 26, 2007 2:15 PM
What about using IP addresses for these resources? Considering that your load-balancing schema works on this level?
Posted by: Phillip at January 15, 2008 5:09 AM
Hello
Really I can not understand how can I apply this rule? should I apply it on "TTL" value that is sent by the application server? or I should ask each user who are using the site to reset that TTL on the IE registery , Please advise.
Posted by: Reham Elgammal at February 1, 2008 5:19 PM
If you have a dedicated ip you can use this in your asset in lieu of the website name, this will also reduce dns lookups on the client.
Posted by: Simon at May 17, 2008 6:06 PM
Comment Policy: We encourage comments and look forward to hearing from you. Please note that Yahoo! may, in our sole discretion, remove comments if they are off topic, inappropriate, or otherwise violate our Terms of Service.
Copyright © 2008 Yahoo! Inc. All rights reserved.
Privacy Policy - Terms of Service - Copyright Policy - Job Openings