Will adding a unique querystring (like ?ver=[timestamp], or even ?[timestamp]) to the file also force the browser to download a fresh copy?

I see that method used frequently - it seems like that might simplify the buildscript - you don't need to worry about the version number, instead allowing the version control system to handle that task.

You wouldn't want to use [timestamp], if that's the current timestamp, because then the file will never be read from cache. The idea of Rule 3 is to increase the use of the browser cache. You could say ?ver=[versionnum], because that will be the same as long as the file hasn't changed, but that's about as difficult as changing the filename. Since its possible that some caches (proxies, CDNs) might ignore the querystring, revving the file name is preferred.

Sorry - I wasn't very clear - [timestamp] would be the time at build, so it seems like caching would be possible with the timestamp.

Is there a reason proxies and/or CDNs could ignore the querystring? Are they just not necessarily built to honor that portion of the URL?

Thanks Again,

Geoff

If [timestamp] is build time that would work fine. I know Akamai can be configured to ignore querystrings, and in some cases for Yahoo! that has been done. Proxies run by private organizations have an incredibly varied (and in some cases non-conforming) configuration, so it is possible.

I use the exact technique for serving all the static files. Although the versioning is made part of the url (a prefix, like a virtual directory with the build number). A Filter is configured on the virtual directory path to process the url. Also, browsers revalidate all the static resources if a "Refresh" or "Reload" button is clicked, so, one thing to note is that if a js file is composed of several small js files (could also be gzipped), the server side should be configured to check for the If-Modified-Since request header to send 304 (Not modified) response header - another level of optimization. (Typically all web servers do this, but if the content is generated dynamically i.e, if the content is not from a physical file then we need to wire this in ourselves). An E-Tag header can also be used for the 304 purpose, but IE has a bug where it does not send the E-Tag back for gzipped resources. One question I have is that the HTTP/1.1 RFC clearly states that Expires header should not be more than an year. So, is it even required to set more than an year in the Expires header or does the browsers happily accept an Expires header more than an year?

The HTTP/1.1 RFC (http://www.w3.org/Protocols/rfc2616/rfc2616.txt) does in fact state in section 14.21 that "HTTP/1.1 servers SHOULD NOT send Expires dates more than one year in the future." This is a guideline and not a requirement, and so browsers do indeed happily accept an Expires header more than a year in the future. Given the frequency with which users clear their cache and fill their cache, setting an expiration date one year or ten years in the future might not make much difference. IE users: what's the oldest "Last Accessed" date of a file (not cookie) in your cache?

HTTP 1.1 specifically states that URLs containing query strings (i.e., "?") are not cacheable, though many (if not most) proxies will indeed cache them. The safest method is therefore to modify the URL, perhaps using a timestamp or version number, since that doesn't involve query strings.

Section 13.9 of RFC 2616 (the HTTP 1.1 spec) says that query strings are cacheable if there's an explicit Expires header, so I believe using a querystring is OK.

My main concern is that you can't guarantee every page of your website will be included in the SERPs. Considering I'm constantly adding new products to my company's website, I need to be sure that customers can find them as soon as possible.http://www.seoptimizerz.com

Given IE's tendency to never remove anything from its cache until the user explicitly empties it, isn't changing the name of your resources every time you make a change (or a new build) to your site just a great way to fill up your end users' disk with files taht will never be accessed again? Maybe you'll make your site a fraction of a second faster by doing this, but if everyone started doing this, I would think that a lot of IE users would start to see their computers gradually slowing to a crawl (more so than they already do, anyway).

Why isn't "Cache-Control: max-age" taken into consideration? Most of the time, it does the same thing as Expires, and overrides Expires if present, which means that if we added an Expires header, it would be ignored by any proper HTTP/1.1 client, but YSlow would consider it much better. :-)

Ideally, we should have both, but still, we shouldn't get dinged significantly for not having an Expires if we have a Cache-Control: max-age.

Also, we use a querystring, simply because it is far simpler to implement for us, and we don't have any CDN problems ... and any proxy that ignores it is simply, and severely, broken and if we can fix that by forcing them to deal with it, then all the better!

What about a Last-Modified header instead? It's hard to predict how much time will something stay the same, but it's much easier to tell when it did change.

http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.29

Ralph, the Last-Modified header is great, but the browser still has to send a request and get a 304 (not Modified) response back. Using a long Expires: header removes even that requirement, so the browser happily caches the resource and never looks back. When you're Yahoo!, even 304s can be overwhelming (I guess).

Yes, I second what Jim said: Expires/max-age are superior as they mean there's no need for a request. However, ETag/Last-Modified is still better than nothing.

This brings up a flaw in the grading though ... if I have no caching, I will get dinged on the Expires header, but get an A for ETag. But if I include ETags, which will not hurt performance and will usually help performance, then my YSlow grade will go DOWN. I think that's ... wrong. :-)

Maybe obvious to Apache Dons but I had to uncomment the following line

LoadModule expires_module modules/mod_expires.so

Otherwise simply adding the following line when I restarted caused a startup error

ExpiresDefault "access plus 10 years"

It should be noted that the "ExpiresActive on" directive is necessary in addition to the ExpiresDefault directive.

drew: IE 6 has problems with orphaned files in the cache that never get removed (there are utilities that can remove them). I don't believe that's the case with IE 7.

pudge: Using "cache-control: max-age" instead of Expires is fine. I prefer Expires because it's more obvious and is supported in HTTP/1.0. YSlow checks for both, so you're not getting dinged. The ETag subject is complex. I really recommend reading the book, as there are 10 pages on these subjects. The summary: the default syntax for ETags is definitely bad for performance. YSlow only subtracts points if you have the default (bad) syntax. If you properly remove the "inode" from ETags for Apache and similarly remove "changeNumber" from ETags for IIS, you don't lose points.

Carlton & Jonathan: Thanks for the additions for how to properly setup the config.

Yes, you can either have Apache cache gzipped files to reduce CPU, or you can do that ahead of time manually. If you work with your CDN you can usually get them to gzip your content. You might have to pay more, since they're spending more CPU time to compress your files when appropriate.

Steve: I don't think that's accurate. I just checked YSlow and clicked to see the headers for files that are dinged for no Expires header, and it even shows the Cache-Control: max-age= header there.

Pudge - If the expires date (either from Expires or max-age) is less than 2 days in the future, YSlow will still subtract points.

Spot the misspelling.

"If your server is Apache, use the ExiresDefault directive to set an expiration date relative to the current date. This example of the ExpiresDefault directive sets the Expires date 10 years out from the time of the request."

Can somebody explain me what's the difference between Expires and Cache-Control? I can't understand the difference between these two...

Can I write Expires code without accessing Apache?
Can I write Expires code in HTML? in PHP?
Can you provide an example for:
a stylesheet
an image
a Flash file
a script
?

Thank you very much.

We have added an expires-header to images, html, css and javascript.
Caching works fine for images, but doesn't seem to happen for css and js, although the expires-header is included in the response-header. Tested from different locations with different browsers and tools (firebug and IEWatch)

Is there anything missing in our configuration/response-header?
http://www.therapie.de/psyche/info/

Hi, Fritz. Your Expires headers are working as expected. I believe the issue is how your testing it. When I tested your page in FF 2.0 and IE 7.0 the components were read from cache on my second page view. Specifically, I loaded your page, and then clicked on the link again. (You don't want to hit "Reload" because that tells the browser to not use the cache.)

One source of confusion is a known bug in Firebug's Net panel where stylesheets and scripts show up, even though they do not generate any network traffic. I'll write a blog about that today on YDN. I'm not sure why that would happen in IEWatch.

I recommend you test it using LiveHTTPHeaders (http://livehttpheaders.mozdev.org/). You should see that on your second page load there are no HTTP requests. I see you put a 1 day expiration date on the HTML page itself. This might be too aggressive, as it could take up to a day for users to see any update to that page.

Hi
Capt newbie here...not really but stumped ow. I know how to set this header from php, its a simple call. But how do I set this for images? I assume something in the .htaccess file.
Could some kind soul include that code here for me?

thanks

If I'm on a shared hosting server, is there a way I can specify I want js and css files to be cached?

I have set my expries headers to be one hour and I still get an F in the rating. Normally people visiting my site don't stay for more than an 20 mins, so I would have thought that setting this time for one hour still saves our servers in the ballpark of 80% of 304 hits. I think this would happen for most sites, yet we still get an F. How far do you need to set this in advance to get a better rating?

Hi, Johno. You're right that setting the Expires header one hour ahead will help users during their 20 minute session. But what about when they come back tomorrow? Then they have to do another conditional GET request. So there are definite drawbacks to setting it too short. Are there any benefits to a short Expires time window? One might think it allows them to change a resource without changing the actual filename of the resource. At Yahoo! we've found there are proxies that don't adhere to the HTTP spec. Our belief is, once you put a resource out in the public, you can only change it if you change the filename. With this guideline, it makes sense to set the Expires date far in the future. The spec suggestions it be no more than 3 years in the future.

As for YSlow, the Expires date must be at least 2 days in the future to avoid losing points.

So just to be clear Steve, do you think I'll suffer headaches if I use a QS instead of a filename? I've been looking through my cache and can see plenty of files from yahoo (gif, js, css, jpg) from yimg.com that use the QS method, yet you suggest in your last comment (September 26, 2007 9:51 AM) that you don't think the QS method is safe?

QS is a lot easier for me, I'd rather just update a conf file in my php than have to re-save my file with a different filename I think. That said the latter isn't the end of the world, if you think there's benefit to it.

PS just got the book in the post, but it doesn't cover the QS/filename debate at all :(

Hi, Seb. Using a querystring should work. You might have misunderstood the comments. I actually said using a querystring is OK, and cited the appropriate reference in the HTTP spec. Good suggestion - I'll mention this in the 2nd edition of the book.

OK, I've worked this into my site now.

To make maintaining this not a complete ball-ache, I've taken every single img/js/css filename out of my code, and replaced them with PHP constants. I then have a file (filenames.php) included on every page of the site, with the filenames in it. Thus if I change a filename, I only have to change it in one place, and not trawl through my site.

The catch is, this means I need to run my CSS through PHP, to correctly translate the echo'd php constants into filenames. As such my CSS file is now main.css.php, with the first few lines as:
--------------------------------
require('global.php'); //this in turn references filenames.php
CacheFor(365*24*60*60);
header("Content-type: text/css")

My CacheFor function is:
--------------------------------
function CacheFor($n)
{
header('Expires: ' . date('D, d M Y H:i:s',time()+$n) . ' GMT');
header("Cache-Control: max-age=$n");
header('Pragma:');
}

The empty Pragma header is in there because my apache is sending a pragma:no-cache out with all php files, and while IE ignores it in favour of the cache-control header, firefox was honouring it. I don't want to change anything in apache, because most php files I do want that header sending out, so it seems easier just to change it manually in my CSS files.

Another thing to be wary of, if for example you change an image that is in your CSS file, you'll need to rename your css file as well, or the browser won't see that it should be requesting a new image name.

I hope to write a short tutorial covering the above and more, if I get around to it I'll post a link up here.

As an aside, I've also realised that my homepage very rarely changes (a splash page introducing you to the site with some options as to where to go) so I've put CacheFor(48*60*60) in that. So if a user re-visits my homepage within 2 days it's 0bytes and 0HTTP requests. Surely that should be an A*? ;-)

PS this is the bit that I was referring to Steve:

"Since its possible that some caches (proxies, CDNs) might ignore the querystring, revving the file name is preferred."

I don't know enough about the structure of the internet and proxies to comment, but I'd rather not risk it, so I'll go with the revving filenames version I think. It just seems more "definite"!

Referring to this comment: "Also, browsers revalidate all the static resources if a "Refresh" or "Reload" button is clicked"

Does not the same hold true when you open a new browser session? I was thinking that IE is aggressive about revalidating everything the first time you hit a site or page in a new browser session. Can anyone verify? Then, when caching is set to Automatically, IE does not revalidate resources for the rest of the session.

Hi, RobD. The "Automatically" setting in IE makes it difficult to do cache testing. You can get an explanation of the heuristics behind "Automatically" from Eric Lawrence, author of Fiddler and member of the IE team here:

http://msdn2.microsoft.com/en-us/library/bb250442.aspx

Search for "The Automatically setting bears some explanation".

The results depend on many variables: Whether you open a new window vs. closing all browsers and opening a new browser. IE vs Firefox. Your cache setting. And the cache settings for each resource.

The "automatic" setting is the common setting for nearly all users. That is why I am testing with it. And, the scenario I am considering is where a user opens a new browser and goes to my site. This is a common case and there seem to be some specific considerations for this scenario. Thanks for the Microsoft link. I will look at that.

This may be a dumb question, but Yslow says there should be an expires header for images, css and script files, Do you just stick a meta tag in each or in the link? Examples please.

Steve, you ask "Given the frequency with which users clear their cache and fill their cache, setting an expiration date one year or ten years in the future might not make much difference. IE users: what's the oldest "Last Accessed" date of a file (not cookie) in your cache?"

But surely shared caches use the Expiry date as well when they decide what to do, and we've got no way of knowing how long they keep our content.

If the shared cache is used by many users, it's still not going to make much difference. If you set it for 1 year instead of 10 years, over the course of 10 years you'd have to make 10 requests instead of 1. That's not going to be noticed across the overall population. This scenario does argue strongly for setting expiration dates that are relative to the time of the request, so it's always rolling out there as far as possible.

How would the expire header affect to the index page of your site. If my index page changes continously, using an Expires header will make the updates very hard to be visible to the world. So looks like the long expiry should be set only for components that are versioned with the file name and html pages that are likely to be updated should have an expiry for one or two days ?

any thoughts here ?

Because HTML pages change frequently, they generally do not have a future Expires header.

Anyone got any good tips on an actual php/apache implementation of expire headers for images? I am on a shared server, so I have little power over server settings.

I checked my phpinfo() and mod_expires is listed among the loaded modules in the apache2handler. I searched a bit, and found that I could set some settings in a .htaccess-file, so I have added an .htaccess-file with the following contents:

#set expires headers to images
ExpiresActive On
ExpiresByType image/gif A2592000
ExpiresByType image/png A2592000
ExpiresByType image/jpg A2592000
ExpiresByType image/jpeg A2592000

Now... YSlow still does not seem to find an Expires header on any images on my site and is still giving me the grade F! (boo-hoo) I notice that developer.yahoo.net's pages on "high website performance" does not have expire headers for their images either according to YSlow, so maybe there is something wrong with YSlow? :) Is there any other way I can test Expires headers on my websites images?

Rails has built in a random number appended to the static file as a query parameter which changes every time the file is changed. Ingenious!

E.g. http://localhost:3000/stylesheets/reset-fonts-grids.css?1185121970

the number at the end changes if it were edited.

This isn't a random number - it's an epoch timestamp, the number of seconds since Jan 1, 1970. In the URL above the epoch time "1185121970" is Sun July 22, 2007 17:32:50 GMT, which should match the last modified time of that resource.

-Steve

I m new to performance tuning of web apps. I would like to set the expires parameter to images, css, js. How can I achieve it with mongrel. We are not using apache or any other webserver.

Thanks,
Vivek

It would be nice if this rule explained somewhere it should expire more than 2 days into the future.

http://www.thinkvitamin.com/features/webapps/serving-javascript-fast

That suggests that incrementing a query string won't make Opera and Safari cache the files. Any thoughts on that?

Hi Steve,
thanks for this great resource. This may seem like a naive question, but in my brief experiments I came to the conclusion that most modern browsers will cache resources anyway (unless you explicitly do a browser refresh). So why would you still go to the effort of implementing this?

thanks
Brian

Hi, Brian. In your testing make sure to try the subsequent page view hours or days after the initial page view. It might be that the browser you're using coupled with the Last-Modified header are hitting an edge case, such as the Automatically setting in IE ( http://msdn2.microsoft.com/en-us/library/bb250442.aspx ).

-Steve

Where do I put that "Expires: Thu, 15 Apr 2010 20:00:00 GMT" header, in the html page itself or in every js, css files? I am new to this performance stuffs. I can write that header with php header function but I have no idea where to put that code.
When I placed that code in html page which was generated from php, the whole page was cached and browser didn't ask for new file even if I change js file names.
Let me know, Thanks.

So how far out is considered a far future Expires header? 1 week? 2 weeks? For some of us, adding version numbers or serial numbers to content is not possible but to expire a page more than a week out really risks that the page will be changed by then and then you're hosed.

I am having a difficult time seeing why you would want to say your page will expire 2 or 3 years in the future. If the point is to increase speed (hence using a cashed version) I could see why a developer would want to put the expiration date in the future, however the risk of having a cashed version being used when fresh content has been added does not seem like a good idea to me. I know I am new at this so maybe I am missing something, but this just seems like a bad idea to me.

I checked out a few Yahoo! web pages looking for a sample of exactly what the "expires" tags would look like and didn't find anything I could use as a precise example. Some pages didn't have any expires on them at all. ???

I was psyched about caching my entire web application until I realized that I may be asking for potential security threats and copyright infringement if the right person analyzed the cached content. I'm not 100% sure but it is entirely possible that caching is a security risk for content management systems, right?

Why isn't anyone addressing the question of how to improve performance if your website is hosted on a shared hosting plan and you don't have access to the server?

In my case that would be a windows server with a website which uses ASP

Please, if anyone of you who own your own web server would look down your aristocratic nose and consider helping one of the riffraff and explain how I can use control caching.

Same problem as someone already reported above. I check and see the expire header but Y!Slow keeps saying there is none. What gives?

p/s: quite a few spam comments in here

Reading through this I see the issue of YSlow saying there is no expires header but it is in fact there.

An example:
URL: http://example.com/icon.php?f=loading

First request:
GET /icon.php?f=loading HTTP/1.1
Host: 192.168.23.51
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Cookie: session=a53a865feb82dc7bbde26d49080d64d7

Response:
HTTP/1.x 200 OK
Date: Tue, 18 Mar 2008 12:08:11 GMT
Server: Apache
Last-Modified: Tue, 18 Mar 2008 00:00:00 GMT
Expires: Mon, 12 Jan 2009 23:59:59 GMT
Cache-Control: public
Content-Encoding: gzip
Content-Length: 1606
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: image/gif

Second request:
GET /icon.php?f=loading HTTP/1.1
Host: 192.168.23.51
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.12) Gecko/20080201 Firefox/2.0.0.12
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Cookie: session=a53a865feb82dc7bbde26d49080d64d7
If-Modified-Since: Tue, 18 Mar 2008 00:00:00 GMT
Cache-Control: max-age=0

Response:
HTTP/1.x 200 OK
Date: Tue, 18 Mar 2008 12:09:10 GMT
Server: Apache
Last-Modified: Tue, 18 Mar 2008 00:00:00 GMT
Expires: Mon, 12 Jan 2009 23:59:59 GMT
Cache-Control: public
Content-Encoding: gzip
Content-Length: 1606
Keep-Alive: timeout=5, max=98
Connection: Keep-Alive
Content-Type: image/gif

As you can see from above the Expires header is set for 12 Jan 2009 23:59:59 GMT (or 300 days in the future). YSlow claims that the expires header is not set.

There is no consistancy in the reporting of no expires header. All of the URLs on my server have query strings and most have the same expires date. YSlow only complains about some.

YSlow also does not complain about some of the pages where the expires header is set in the past for components (images/javascript/html) which should not be cached.

Any ideas?

Hi Steve,

I have tried to search on net a lot to set expire header for js/css and images but i did not get anything. I have also gone through complete discussion regarding Enxpiring but still no one has replied for issue regarding js/css/image Expire setting in jsp.There is no use of YSlow Add an Expires Header rule if we are not able to get solution of it.Everyone is eager to get A for Add an Expires Header but they dont know how to solve it. It will be really great if someone can guide to set expire time for JS/CSS/Images for JSP.

JSP code is highly appriciated.

Regards,
Jigar

hi Jigar,
If you are using TOmcat the below is the way to add expiry date to the
js,images and css files that i figured out.

1)you need to create a filter for the above files to add an expiry date.
At the below link you will get code illustration of how we should do it
------------------------------------------------------------------------
http://www.jguru.com/faq/view.jsp?EID=1311010



2)How to make sure that the expiry date is added properly to those files?
--------------------------------------------------------------------------
I did a lot of investigatation on the above problem. you can use the following
tools
1) Yslow itself to check which compenents has expiry header and which components do not have.But it is some what buggy is not correctly reporting the expiry date of components.
2) http://www.webpagetest.org
AOL's Page test tools you can download the desktop version and for accurate picuture you can check the optimization report to know which components have expiry header which components do not have.This tool also some gives different problems.

3) The best way that i figured out is

The number of httrequests that will go to the server is equal to the number of components (js,img,css) present in the webpage.

Step1) write some debug messages( i.e printing requested url and filename) in the filter that you have written for adding expiry date.
Step2)clear cache and download the page from the browser now check whether filter has modified any response by looking at the debug messages.
The number of debug messages = number of components requested.
by debug messages you can know to which component we have added expiry header.

Step3) Load the page again (please don't refresh as per my understanding refresh will invalidates all components of the browser cache and brings fresh components from the server) by copying the url and pasting in another browser window.

Now at server you can observe that no debug messages has been printed from the filter which means that those component have not been requested from browser because they are present in the browser cache which means our expiry date is working properly.

i hope you understood the above.

regards
rama

Is there any suggestions on how to configure Websphere to set the expire header for css, js, etc. files. I tried what was suggested above in the link: http://www.jguru.com/faq/view.jsp?EID=1311010 but was not successful.

Regards,

Marty

I was wondering if you had looked at other cache-control headers such as 'public', 'private' and 's-maxage' and their effect on the caching performance end to end?

Hey,

For served static content I added the expiration headers, both Expires header and cache-control set to one year. When having the last-modified header ON I noticed the browser makes the If-Modified-Since request.
When removing the Last-Modified header but leaving the expiration headers, in order to get rid of the redundant request, the browser downloaded the static files allover again for each page.

Response Headers
----------------------
Date Sat, 26 Apr 2008 12:29:08 GMT
Server Apache
Accept-Ranges bytes
Cache-Control max-age=31536000, public
Expires Sun, 26 Apr 2009 12:29:08 GMT
Vary Accept-Encoding
Content-Encoding gzip
Content-Length 12489
Content-Type application/x-javascript

Is there a way to make the browser use its local cache for a whole year without validating(using If-Modified-Since)? Isn't it what "Expires" header mean?

Thanks in advance,
Aviel.

Hi,

I have it all set up perfectly throughmy .htaccess file.
BUT there are few images and thumbnails on the page that are called from external sites (e.g. YouTube thumbnails).

Is there any way to set Expires 2 years on these *external* images???

I tried adding type="image/jpg" , thinking that then it would obey the htaccess Expiration settings. But nothing seems to change. Is there anything else that I could try?