The change number can actually be synchronized under IIS by setting the MD_ETAG_CHANGNUMBER property to 0 under /LM/W3SVC for all servers in your farm. Here's a URL with some good information on this setting and some associated KB articles. Under IIS6 on win2k3 SP1, I had to manually add this to the metabase as attempts to do it with adsutil and metabase explorer failed.

Sorry. Forgot the URL:

http://dotnetjunkies.com/WebLog/leon/archive/2005/02/16/54630.aspx

This is a little dated but is a good start in understanding the problem.

Since Apache 1.3.23 The Apache FileETag can be set to override the default and only use modify time and size only (e.g. set FileETag -INode in a sub-folder containing only images) so that the same static object across cluster hosts can have the same etag.

Agreed, and this is from my config even though the site gets an F for ETags:

FileETag MTime Size

So what's up YAHOOO????

Eleven of the thirteen components on the page have ETags, yet I got an F. The developers of YSlow must have went to some tough school.

"Eleven of the thirteen components on the page have ETags, yet I got an F. The developers of YSlow must have went to some tough school."

That's because you're reading the answers wrong (or not reading it all).

YSlow is saying that ETag = BAD, not ETag = Good. The more ETags you have the worse your score.

Personally I have a problem with this because there are some cases where ETags are bad, but if you've mitigated those cases (and there are many ways of doing so), then there are cases where ETags are good. To blindly say they're bad without knowing the background is a problem... but then YSlow can only give you a hint and you have to analyze your own results to see if they're accurate.

Is there anything I can do in the HTML to improve the yslow score? It seems like the score is based on my server settings. That's something I don't have access to.

ETags are actually fine to use in a load-balanced server setup when the data is served from the same NFS share across all servers. I don't think giving an F for using ETags makes much sense, since it actually depends on the context whether the usage is good or bad. YSlow should probably give a warning if the page components use ETags, but it shouldn't be graded.

P.S. This commenting system is broken. This is my first comment, and I'm getting an error saying "Too many comments have been submitted from you in a short period of time.". Bummer.

I set "FileETag none" in my ".htaccess" file, but my ".css" file still loads with an Etag. Doos the FileETage setting apply only to html files by default? How do make it apply to my css file?

This link http://httpd.apache.org/docs/2.2/mod/core.html#fileetag would be helpful in the article.

For those of us who have tuned ETags and use them for real caching, it would be great if YSlow recognized the default patterns for IIE/Apache. For example, if the server sends the header combo of Server: Microsoft-IIS/* & ETag: "23ecf7e5da1c71:9a7", or Server: Apache/* & ETag: "646c033-4291-468adbc6" then flag it as an issue. Presumably if someone has created another format then it is using custom logic. It shouldn't rate an 'F' for ETags providing valuable caching information.

I'll second those who say that it's troubling that YSlow assumes ETag = bad. Every time we use ETag on our site, it works just fine, either because we are using reliable information across multiple servers, or because we're not using multiple servers.

Any idea on configuring ETag in Tomcat? Either remove it, or tweak its computation setting to only account for timestamp and file size?

If you're using Tomcat to serve static resources, you have other problems.

Anyway, I'm going to take my grades to the department chair ;) I agree with the comments above that YSlow's "F" for using ETags at all is suspect. Using
FileETag MTime Size
in Apache sufficiently shelters ETag calculation from the host-specific inode differences. It'd be more clever of YSlow to drop and re-open connections repeatedly for the same resource that is returned with ETag headers. If the the resource is returned multiple times (same meaning the MD5 matches) but returns different ETag values from one request to the next, that's a definite "F". Returning the same ETag for the identical resource should earn you an "A".

A few observations:

First, for the vast majority of websites, this eTag thing is a non-issue. It only becomes a problem if the site is served by multiple servers. Therefore, I agree with previous posters that the grading in YSlow is a bit tough on this feature.

Second, the information provided above regarding IIS is a bit confusing-- the link in the post is about IIS 5, and although the text says it describes how to remove ETags, it actually doesn't-- it just describes how to synchronize multiple servers so they generate the same Etags.

The dotnetjunkies link in the second comment is more helpful, but it is also confusing because it talks about and links to a hotfix that has to be specially requested from Microsoft. This information is outdated-- that hotfix was included in Windows 2003 service pack 1.

If you are using Server 2003 SP1, it appears that all you need to do to make eTags work as expected on your cluster is to make sure that all of our servers have the same MD_ETAG_CHANGENUMBER in their Metabases.

Furthermore, if you're using the iiscnfg.vbs script (with the /copy switch) to keep your cluster configuration synced up, this keeps the MD_ETAG_CHANGENUMBER synced up too. I personally can't imagine running an IIS cluster without syncing up the IIS metabases, but I guess some people do it.

If I'm wrong about any of the above, please post here and tell me so.

I checked out the Firefox config in about:config and found a setting for ETags.

extensions.firebug.yslow.pointsEtags set to 0 will report A grade performance even if ETags are used on the site.

Thanks for all the comments! Yes, this was the most controversial chapter during tech review. There are 7 pages on this topic in the book. The summary:

If you only use one web server, don't worry about this rule.

If you have more than one server and have configured ETags to do something special, that's fine. But if you didn't change the default ETag syntax in Apache, IIS, and lighttpd, ETags are bad for performance. If you configure your ETags to remove "inode" and "changeNumber" from the ETag, it's not much more valuable than the Last-Modified, header, so just remove them and save those bytes from the download.

I'd love to hear cases where the power and flexibility of ETags are being used. If there are many of them out there, I could change YSlow to check the ETag syntax before subtracting points.

Hi Steve,

so will you be changing YSlow to not give me a bad score for using Etags. We're using it correctly across multiple servers.

This (and not using a CDN, which isn't realistic for us) are the only things standing in the way of the perfect score.

ta.

>> I'd love to hear cases where the power and flexibility of ETags are being used.

Hi Steve,

For somethings however, like js files or css files composed at run time (multiple files combined in to a single file), ETag might be a better option as the aggregated file is not a single file and we do not have a Last-Modified date from the file system. (Well, we can definitely work around that using a timestamp from a config file and manually updating that timestamp value when a file actually changes etc). But I have seen an issue with Internet Explorer where it does not return ETag for gzip resources like js and css.

I would like to know what is the power and flexibility provided by ETags from Last-Modified date. Like you said, we can as well use the Last-Modified date. The only two issues that I see with using Last-Modified date is that 1) This is on the assumption that client would not intrepret or change the header and send a different value in the If-Modified-Since header. If a client does then using the Last-Modified header would not be helpful 2) The resource does not change in a split second between two requests from the client. Well if that does then there is a risk of the client still getting a 304. But for all practical resources likes images, css, js, rss feeds etc, I think assuming that a resource does not change in a split second is fine. So, I would like to hear about the things that ETag provides which Last-Modified date does not.

Also using a CDN (as per your second point in this series), wouldn't all this (that is setting appropriate cache headers, Last-Modified or ETag and sending 304 appropriately) be taken care by a CDN?

For those of you that run sites on IIS in a load balanced configuration, this might be of interest to you.
On behalf of a client, I've looked far and wide, and the only solution I felt comfortable implementing, was to use ServerMask to remove the header. http://www.port80software.com/products/servermask/iis

Anything else involved a lot of things that could go so horribly wrong, that I didn't have the stones to try and implement them on the clients production servers.

Similarly, I use ETag for dynamically generated RSS feeds. We *could* store a last-modified somehow, but that is problematic for various reasons. Easier and simpler to provide an ETag.

Hi Guys,

So how do you remove eTags in IIS?

Steve, I've been slowly developing my own Wiki engine and I have been paying a lot of attention to caching and other performance issues. My software generates Etags from the last revision number of a resource (page, etc). YSlow correctly shows my wiki site caching very well, but gives me an 'F' because of the incorrect assumption that Etags are always bad.

Personally, I feel that properly constructed Etags and last-modified headers are a much better approach to caching content than the silly nonsense with "far future" expiration and putting version numbers in filenames/URLs. I'm left wondering if those measures are really just to get around buggy clients and/or proxies.

Ed - What do you mean by using ETags correctly? What data are you putting in the ETag that makes ETag validation superior to Last-Modified validation to justify the extra HTTP headers?

The examples of good reasons to use ETags are appreciated. Dynamically generated JS and CSS files, and RSS feeds are good examples.

Ian - In the latest version of YSlow (0.7.1) your customized ETags should no longer be losing points. A far future expiration is _much_ more important than ETags. A future expiration date means there is no HTTP traffic, period. A good ETag means you'll still have a conditional GET request on (nearly) every page view.

As a note, for Rule 2 you can add your hostnames to the CDN config setting so you don't lose points there (http://developer.yahoo.com/yslow/faq.html#faq_cdn). If you do use a CDN, their default practices might not be optimal. You could work with them to change the settings for your content.

Steve,

Maybe you could check one of the images of the site or something a few times. If the ETags are the same then it's because it's properly used. If the ETags are different then you know it's a cluster with the default settings.

Wouldn't this be accurate for the ranking on YSlow?

In YSlow 0.7.1, the only way to lose points is if the ETag has the default syntax (inode-size-timestamp for Apache, Filetimestamp:ChangeNumber for IIS). The default syntax is bad (for sites with more than one server) because it contains information that varies across each server, which defeats browser caching. If the ETags are the synchronized to be the same across all servers, then the additional information in the ETag is minimal - the Last-Modified header already has the file's timestamp. So it's still inefficient to send an extra HTTP header that has little value. It's best to either use ETags for a more advanced validation scheme, if needed. If you don't need an advanced validation scheme, it's best to simply remove ETags.

> The examples of good reasons to use ETags are appreciated.

Search pages: the ETag for result pages can be generated using hash(search query)+last_modified(relevant databases).

Hi Guys,

How do you remove ETags in IIS? Is it a setting? What am I supposed to be looking for? IIS 5.1 and IIS 6.0 instructions would be good

Cheers

Please search IIS support sites for information on IIS. Here's one link I found:
http://blogs.msdn.com/yanno/archive/2006/03/03/543176.aspx

-Steve

Having read the story behind ETags, I'm trying to figure how I should deal with the ETags in combination with a CDN network, in a situation where data may change every day (or multiple times a day). Considering the fact that the a CDN server will request the data from the origin servers as soon as it's requested on that particular server, it's stored on that server on some inode, with a certain filesize and timestamp(the time at which the file was requested from the origin server?). But if the CDN cache get invalidated forcing to retrieve the data from the origin servers again, a file that might not have changed will only have the filesize as unique property and inode number and timestamp will change.
My conclusion would be that ETags in combination with a CDN network for hosting appears to be useless. Can somebody give some more insight on this topic?

If you're not using ETags for some specialized form of validation, it's best to remove them.

> Search pages: the ETag for result pages can be generated using hash(search query)+last_modified(relevant databases).

Not really. Last-Modified does just as good job there. The search query is in the URL so no need to reflect it in the ETag.

Rafal and Henrik - Those are great idea about using ETags to cache search results. Henrik is right - if you put the last_modified(relevant databases) in the Last-Modified response headers, and you have the search query in the URL, that would work. However, the last_modified(relevant databases) could change frequently, and not necessarily invalidate the search results. Instead, it could be checksum(search results)? If the checksum of the search results just generated matches the checksum of the cached search results, a 304 response could be returned, reducing the size of the HTTP response.

The performance of a site would definitely degrade if using Etags on a load balanced web site with tags not being synchronized in IIS blah blah .. we all know that.

But I feel that in most cases a max-age header or an expires header would do better because even though etags are used to validate freshness, there is still an unnecessary round trip between the client and the server(304 response). Properly defining an expires header would eliminate a turn between the client server improving performance by using true cache over especially over high latency links as mentioned by the author.

"Even if your components have a far future Expires header, a conditional GET request is still made whenever the user hits Reload or Refresh."

Also the MS link posted only synchronizes the E-tags coming to my question .. is there a MS article to disable E-tags besides writing code for it?

E-tags discussion has got be a bit complexed.

Our environment is clustered.
My test will be enabling ETag and eliminate the inode element in the identification of the ETag. As I understand it, the ETAG will now be the same on all servers.
We are using Silk Performer for our high volume tests.
Does this approach make sense or is this scenario provide little value.

Per my comment on July 28, I don't think the benefit of ETags with just timestamp and filesize justify the costs.

"If you're using Tomcat to serve static resources, you have other problems."
Not necessarily. You might be using jsp pages just to improve maintainability, eg to insert standard headers, footers etc. What you end up with is static in the sense that the same URL will produce the same content, so it's OK to put Last Modified (or Etag) headers on it.

Hi
Can anyone help me on this , I really do not know how can I configure Etags on IIS? please advise,

My experience in testing the ETag in Apache 1.3.xx is that the Directive
FileETag None
Doesn't function at all. The ETag Header is sent anyway. It does work in Apache 2.x though.

So the only solution to Apache 1.3.xx, since we are using clustered servers behind a load balancer,
is to set the ETag to the file size, which should be the same on all clustered servers anyway.
FileETag Size
The 'Last-Modified:' header would then be used for the date validation and 'ETag:' header for the size. Well I haven't tested the idea yet but I'll get the results son.

mdutil get w3svc -prop 2039
mdutil set w3svc -prop 2039 -value 768782

Programatically set MD_ETAG_CHANGENUMBER in IIS:
"C:\Program Files\Resource Kit\expand.exe" -R Windows_2000_Disk\I386\MDUTIL.EX_ .
MD_ETAG_CHANGENUMBER Metabase Property (IIS 6.0):
http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/ef7f9d58-2a96-4bd8-8ac1-2a67b43284f1.mspx?mfr=true
http://support.microsoft.com/kb/922733

Hi Steve,

I have just a remark on your point that "removing the ETag reduces the size of the HTTP headers in both the response and subsequent requests". Right now, we try to conduct some performance optimizations for our midsize Web site. Today, I had a discussion with our site admin concerning my proposed optimizations and after a long argumentation he convinced me that it does not really make a difference whether to leave fixed ETags in or take them out because an ETag itself takes about 30 bytes in the HTTP request/response. And as the HTTP request/response data is always packaged in a TCP packet which has a size between 1460 and 1500 bytes (I think) the existence or non-existence of the ETag will in most cases make no difference to the overall performance or transfer volume. I am not sure whether this is 100% right, but it makes sense to me.

Cheers,
Martin

Hello there,

i need to turn the etag off. we use tomcat 5.0.
Is there any option in server.xml or web.xml?
i get the etag by calling static content.
for example a *.txt file in the root of the
webapp.

thanks

Hi Steve

Since it seems very difficult to remove etags from the IIS then maybe the program could be change to not lose points when the etags has been configured to be the same on all servers? When reading the Microsoft recommendation this will change the etag to something like this: "083cbb3d97c51:0" (ending with ":0")

BR
Esben

Hi (again) Steve

What happens if you upload an older version of a file to a server without etags? I thought that the etag was superior to the "last-modified" header because it also checked older files. So for instance if an image was rolled back to an earlier version the etag would make sure that the user now gets the older version?

By the way thank you very much for a great program and an even greater book! I have really been able to optimize our customers sites a lot! :-)

:-] Esben

Hello to all !!

I am a newbie to web development and I feel good to come to know about web performance rules at a good time, before I would scratch my head with slow server speed.

I had been reading the yslow performance rules and i did make the changes required, which helped me improve my grade n mark.

I had a D with 68 points yesterday and after implementing the smartoptimizer, adding CDN of my domain and removing Etags by directive FileEtag None, helped me get 88 score with a B.

But my problem still lies there as , the yslow still shows an F for CDN, and it shows links of images which my page gets through photobucket. I added www.photobucket.com to the CDN host names, but it still shows the same error n grade.
The same is for ETAGS, it shows the image links that are fethced from photobucket and says these images have an ETAG and gives me an F grade for it.

Any suggestions please !!

Thanks in advance !

Mandeep

To Martin B - tcp packets are variable in size, usually for modems the maximum size is 1460 to 1500 bytes in size. A reduction in header size will reduce the tcp packet size. It is quite possible to have a tcp packet with a 1 byte payload, and it will be much faster to transfer this packet than a tcp packet with a 1400 byte payload.

I had been reading the yslow performance rules and i did make the changes required, which helped me improve my grade n mark., it seems very difficult to remove etags from the IIS then maybe the program could be change to not lose points when the etags has been configured to be the same on all servers,
http://www.cyberdesignz.com/ , you can get more information from this.

It looks like people are still struggling with this, here's a simple way of disabling ETags for your IIS6 website. From the website properties, select the HTTP Headers tab and add a custom header called Etag with a blank value. That's it. No metabase editing or app coding required.

Hi Team,

I"m looking at ETags concept to improve the performance. Please share your knowledge on below queries.

1. Where we have to specify ETags in the application?
2. How to test whether ETags are working properly or not?
3. I observed some sample code in http://developer.yahoo.net/blog/archives/2007/07/high_performanc_11.html ; How to generate
ETag: "10c24bc-4ab-457e1c1f" code?

Please respond as early as possible to krishnanvr@aditi.com

Thanks,
KVRVR

If you happen to use Nginx for proxying/loadbalancing, and for some reason don't want to modify your backend servers you can instead have the proxy delete the ETag header before it sends the response back to the client: http://wiki.nginx.org/NginxHttpProxyModule#proxy_hide_header

I've got a strange issue, where in apache2, I'm using

SetOutputFilter DEFLATE
FileETag MTime Size

and I get my content gzipped and etagged (all good there) but YSlow reports that the content is NOT etagged... I'm wondering if it's because of the "-gzip" at the end of the etag:

Etag: "11f7c2ec-451f4-46bdd845271b9"-gzip

Anyone heard of this?

I'm getting the same ETag detection problem as Alan. I think that YSlow detects an ETag with three(+) parts as incorrect. BTW I think Alan's ETag rule is still incorrect as per YSlow's rules; it is still using the INode (it has three values in quotes, not two). You have to explicitly remove the INode like this:

FileETag -INode

With GZIP,

Etag: "2273-4717d05dc0a80"-gzip

http://httpd.apache.org/docs/2.0/mod/core.html#FileETag

Steve,

I really can't figure out how and where the ETag configuration is made in apache. Could you tell me if ETags are turned on by default or if there's some directive in Apache/Tomcat conf file that turns the ETag on?

Thanks.