What do you suggest for mod_gzip_minimum_file_size ?

Anything under 1K is probably not worth gzipping. I believe the default is 0.5K, which is fine.

Is it ever the case that users with slow computers but fast internet connections actually get poorer performance as a result of gzip? That is, the time required for decompression is greater than the extra transport time for an uncompressed file.

In my experience gzipping files > 1K is faster than not gzipping. It could be possible for gzipping to be slower if the web server was overloaded, the client CPU was overloaded, and the Internet connection was very fast.

QUOTE 1: The only other compression format you’re likely to see is deflate, but it’s less effective and less popular.

QUOTE 2: Apache 1.3 uses mod_gzip while Apache 2.x uses mod_deflate.

So, if you want to use the more recent version of Apache, you are stuck with the less efficient, less popular compression type ?

Despite its name, the mod_deflate module uses gzip for compression. So you're good to go!

Gzip compression is very good if you need to safe bandwidth, but don't forget you have higher server load and you need top have a bigger machine.

Is there any problem with firefox requesting compressed javascript? I'm using mod_deflate, and I verified it is working with

curl http://myurl.com/javascript.js --head --compress

.. but all the javascript files still show up under #4 on yslow.

Interesting .. yahoo seems to be doing something else with it's javascript files (note, no Content-Encoding header):

$ curl http://us.js2.yimg.com/us.yimg.com/a/1-/java/promotions/js/ad_eo_1.1.js --head --compress
HTTP/1.1 200 OK
Last-Modified: Thu, 06 Apr 2006 21:17:00 GMT
X-Yahoo-Compressed: true
Content-Type: application/x-javascript
Cache-Control: max-age=298038277
Expires: Tue, 03 Jan 2017 03:35:48 GMT
Date: Wed, 25 Jul 2007 15:11:11 GMT
Connection: keep-alive
Vary: Accept-Encoding

I've read you can also pre-gzip the static HTML files if you use Apache: the files are gziped manually and are served by default by the web server.

The server only send non-gzipped files if it realizes it is talking to an older browser that does not support gzip. In that particular case, Apache can either (i) gunzip the files on the fly or (ii) be smart enough to send the corresponding non-zipped HTML file (I believe using mod_negotiate but I've never tried).

In theory, this method is great because it consumes a lot less CPU power. what do you think of this method? I'm talking about static files only.

Compressing content will conflict with using CDN's. Not all web services support compression. For example, Akamai will not cache the content since it would need to keep two copies, one compressed and one uncompressed. The speed of the servers is helped by not having to make any decisions - just serve up the content requested.

Is there an absolute bonehead guide to gzip some where? I've looked but most of the tutorials assume a lot more knowledge than I have.

YSlow seems to report that a website is not using gzip even if it is if the browser uses a cached version of an object. For example, if you use a far future expires header and gzip compression to serve a CSS file, the first time the browse requests it, YSlow reports an A for gzip but the next time you refresh the page, the CSS file shows up as not gzipped (since its coming from the cache).

I'd be nice if Yahoo! shared the secret to compressing js/css safely across many browsers. Versions of IE before IE7 have quite a bit of trouble handling compressed js files.

Is there an apache/lighttpd module that allows for conditional compression based on browser version? Flat out compressing every js/css is dangerous, especially if you want your site to work cross browser.

Does anyone know the procedure(steps) for creating mod_deflate.so file(module) in apache 2.0.47 for windows..?
your help is very much appreciated.

Does anyone know the procedure(steps) for creating mod_deflate.so file(module) in apache 2.0.47 for windows..?
I hope this mod_deflate.so module will help in the apacheWebCompression(like gzip).
your help is very much appreciated.

Is there any recent data on A) the percentage of user agents which still break on gzipped js/css or B) best practices for preventing this problem?

These instructions were a big help for installing/configuring this on Mac OSX 10.4 Apache:

http://www.truerwords.net/articles/web-tech/serving_compressed_with_mod_gzip.html

These instructions were a big help for installing/configuring this on Mac OSX 10.4 Apache:

http://www.truerwords.net/articles/web-tech/serving_compressed_with_mod_gzip.html

Using curl to view HTTP requests might not be accurate, since it doesn't send all the headers a true browser sends. In the case of Yahoo, we only send gzipped content if we see a valid User-Agent header in the request (which curl does not do by default).

Avoiding gzipping on the fly is important. Apache has several configuration settings for this.

You can work with your CDN to see if they can change their settings to support compression.

My book (out on Sept 10) has an entire chapter on this with recommended settings to handle the various browsers.

HI there,

A stupid question.....
How do we gzip files?
i mean what are the steps followed or what is the syntax for performing GZIP.
Thanxxxx........

'gzip' and 'deflate' are different names for the same thing. Basically the specification used to call this gzip, but it was pointed out that's actually the name of a program, the GNU zip program, which implements an algorithm called deflate. So newer versions of the standard allows HTTP servers and browsers to refer to this algorithm as either 'gzip' or 'deflate'.

When someone says "gzip is slower" or "deflate is bigger" or any similar nonsense they don't know what they're talking about. Unfortunately this particular Yahoo article seems to be just such an example. But, even if the description is wrong most of the advice is fairly accurate.

Hi, Nick.

If we limit our discussion to Apache, it is true that mod_gzip (Apache 1.3) and mod_deflate (Apache 2.x) both use gzip. The reference to deflate, however, is in the context of the deflate algorithm (used by other web servers), which is based on Huffman coding and LZ-77 compression. Whereas, gzip is a different algorithm (although it includes a variation of the LZ-77 algorithm).

In summary, the modules mod_gzip and mod_deflate both use gzip. The gzip algorithm and the deflate algorithm are different.

What about browser bugs with gzip? Are there any browsers that we should not serve gzipped content to?

Yes, there are edge cases. You can see more details here:
http://support.microsoft.com/default.aspx?scid=kb;en-us;Q313712
http://support.microsoft.com/default.aspx?scid=kb;en-us;Q312496

You can implement a browser whitelist approach in Apache 1.3 with directives live this:
mod_gzip_item_include reqheader "User-Agent: MSIE [6-9]"
mod_gzip_item_include reqheader "User-Agent: Mozilla/[5-9]"

Does anyone have any IIS implementation suggestions for limiting gzipped content to internet explorer browsers, specifically versions less than 6? ie 5.5, i.e 5, etc. (I wish we were on Apache but that's not my decision to make)

Mujahid: http://www.gzip.org/ suggest Windows users use 7-zip or PowerArchiver - I got that homepage from ol' faithful: http://en.wikipedia.org/wiki/Gzip :-)

hi
i don't know how to gzip some css and js scripts
i need somebody to show me how it can be done
please, give me a practical example
thanks

Here's an example for Apache 1.3:

mod_gzip_item_include file \.js$
mod_gzip_item_include mime ^application/x-javascript$

Apache 2.x is different. This is covered in Chapter 4 of my book. Here are links to the relevant Apache documentation:

http://www.schroepl.net/projekte/mod_gzip/
http://httpd.apache.org/docs/2.0/mod/mod_deflate.html

For those worried about extra CPU load due to zipping lots of files on the fly:

A long time ago I created an Apache 1.3 module which, upon a request for "foo.html" from a compliant browser, would check for the existence of a "foo.html.gz" file with a newer timestamp, and if found, would serve it. If not found, it would just serve the "foo.html" version uncompressed. A simple script run as a cron job would then traverse the system in parallel, compressing files as they are spotted using the command-line gzip.

That approach worked extremely well for a high-load site I was working on at the time, but I'm not sure if the approach is now out of date... I could update the code and release it for Apache 2.0 if people would find it useful.

It all comes down to whether CPU usage on web servers is your bottleneck I suppose -- which is not the case a lot of the time.

Anyone interested?

Brendan.

somebody help me
I don't know how gzip js and css script
I have apache 2.2

@Brendan - That would be awesome

I've setup apache 2.2 with mod_deflate. It's working, but won't compress anything under 2k. I don't see a way to set the minimum file size like you can with mod_gzip.

I bought your book expecting more insight into mod_deflate configuration and compression techniques, but it's pretty much just what's on this blog and the apache site.

Search for "DeflateMinLength" - you should find some handy references.

The main benefit of the book wrt gzip is the discussion of the edge cases. There are a lot of tradeoffs involved and there's no easy answer. I hope you found that part of the chapter (and the rest of the book) helpful.

-Steve

I am using Apache/2.0.49, RH AS3.
FireFox 2.0.0.7

I am using mod_deflate
SetOutputFilter DEFLATE
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary

I keep getting "illegal character" on a attched .js. "illegal character" doesn't on any particular .js. Its reporting randomly on multiple .js files.

Any idea/help ?
--Sriram

DeflateMinLength only seems to work with the hacked version created for apache 1.3, not the real version for apache 2.2. Seems there are a bunch of other configuration params for the 1.3 version, just doesn't work for mod_deflate for apache2.2.

Yslow gives me errors on a JS file that is 2.2k because it thinks it could be compressed, but using mod_deflate for apache2.2, it doesn't seem possible to get it any smaller.

Guess the solution could be to make the js file bigger, then it would compress it :-)

Thx Steve!

Dave - Ugh, that's a bummer. I'll enter a bug for YSlow to work around this. In the meantime, would you be willing to log a bug request with the Apache team to add that to Apache 2.x?

-Steve

I did a portupgrade to a test Apache2 server and it bumped my grade 'A' site to a B. YSlow is showing that my .js files are not being gziped. YSlow shows css and html gziped fine.

New version is Apache/2.2.6 (FreeBSD), PHP 5.2.4 with eccelerator 0.9.5.2

Yep. That's my problem as well also. I'm running apache 2.2 and no matter what I try it doesn't (or Yslow says it doesn't) gzip the js files. thought it was a min size issue but doesn't appear to be.

I tested to see if my js files are actually being gzipped and they are according to this site: http://www.port80software.com/products/httpzip/httpcompression

They have a compression tester on the right hand side of the page.

Is this an issue with Yslow and apache 2.X/mod_deflate?

Here is a site running Apache 2.3 that has .js files under 2K that are gzipped:

http://beehive.apache.org/

My sources say "mod_deflate hasn't change much between 2.2 and 2.3-dev." Could it be something in your apache config?

Hi Steve,

After testing on several different PCs I get different results. One PC says the file is gzipped, and one does not. I tried uninstalling yslow and firebug and reinstalling with the same result. The file IS being zipped, just yslow isn't picking it up.

I've seen other situations where Yslow does acknowledge expires headers under the performance tab, but doesn't show the expires header date under the components tab.

Check out http://www.cincinnatibell.com/ for an example of that. It's running IIS 6, with a 14 day expiration of images,css,js,etc. One site I'm trying to get from an F to an A.

Thanks again!

Hi, Dave. These are separate topics. People have reported that, unbeknownst to them, other software on their box (Norton?) was causing resources to not be gzipped. That could explain why you're seeing different results on different PCs.

Wrt Expires headers, what you're seeing is correct. Those components contain max-age, which is an acceptable alternative for caching, so they are not penalized under Rule 3. But they have no Expires header, so that's why none is shown in the Components view.

Thanks for your help, Steve. I checked the headers out with a utility for IE called IE Watch, below is a link to a screenshot showing that files do have an entry for expires, but is there something else missing? Is that expires actually translated as max-age?

http://s3.amazonaws.com/cbt_files/expires_header.jpg

Can't seem to find too much info on this for IIS. Apache is super easy to setup.

thx!

Hi, Dave. The headers you're looking at are for the 304 Not Modified response, which can't be cached so the Expires headers are irrelevant. Clear your cache and repeat the experiment. For the 200 response you'll see that there is no Expires header. Which again, is fine since max-age is an acceptable alternative.

Steve,
If we already compress (gzip) at Page level, do you have any stats on how much gain there is to additionally compressing at object level (js, css etc)
Do you know of any downside to GZIPing within your Weblogic server cluster, vs doing at apache level?

Steve ,
Did you guys ever consider that mod gzip actually taking more cpu and creating load issue to the server rather then.

I also see in one of the header posted above that Yahoo has Keep alive on. Is there any downside of keeping the connection alive.

Thanks
WordPress Guru

One of the readers above comments about the ability to pre-gzip, thus avoiding CPU overload on your web server. Apache's modules have options to help do this, as described in the links in the original article. The chapter in my book talks more about this.

If you enable Keep-Alive you must have enough socket connections for all your users.

re: dave on October 22:
>I tested to see if my js files are actually being gzipped and they are
>according to this site:
> http://www.port80software.com/products/httpzip/httpcompression
>Is this an issue with Yslow and apache 2.X/mod_deflate?

Steve, I'm seeing the same thing on my site, http://www.trenchmice.com/. YSlow reports flipform.js as not being Gzipped, but port80software.com's checker says that it is! I ran both from the same PC, and same browser window (Firefox 2.0.0.9), multiple times within a few seconds of each other.

Could you confirm that YSlow doesn't always correctly detect a Gzipped object? If not, then I have a _weird_ problem on my site, as all my other JavaScript objects are reported as deflated, but not this one. Plus it would mean there's something doubly-odd with port80software.com.

I love YSlow BTW! It's the first client-side perf tool I use on a page. Thank you!

Hi Steve,

You say "The gzip algorithm and the deflate algorithm are different", but RFC 1952 ("GZIP file format specification version 4.3") itself says "The format presently uses the DEFLATE method of compression". The wikipedia "DEFLATE" article and several other RFCs agree with this.

I did find a page which explains the difference: "deflate" is RFC 1951 ("DEFLATE") plus headers from RFC 1950 ("ZLIB 3.3"); "gzip" is RFC 1951 plus headers from RFC 1952 ("GZIP 4.3").

So while I understand that "Accept-Encoding: deflate" is less popular, I'm not sure how it's also "less effective", since it's just different headers on the same compressed data.

My test served the same file through deflate and gzip, and showed that gzip had a smaller size. Perhaps it has to do with the default levels of compression (IIS vs. Apache).

I agree with Tim with respect to the contents of RFC 1950, 1951, and 1952 as well as the references thereto from RFC 2616. The misnaming/usage are also covered in the faq of the authors at zlib.net.

The fact that you got a different size is due to factors other than the relative efficiency of gzip versus zlib. Especially as the zlib format has smaller headers and both gzip and zlib are wrappers around deflate.

The gzip headers contain much information that is irrelevant in this usage.

Note that the authors of RFC 1950, 1951, 1952 are also the authors of the underlying library. Given that fact, it seems their information would be accurate.

Given the definitions in the mentioned references, the only thing that your testing indicated was that the settings used in the tests were sub-optimal for the testing goal.

Thanks for the great background, Tim and Spenser. The statement that deflate is less effective is based on the fact that the same files delivered with deflate are larger than when run through gzip. It would be great to determine where this difference comes from, the compressed data or headers, etc., but the point I was trying to make is you'll typically see a smaller delivery size using gzip.

How do you gzip your components? Do you have to have root access to your server or can you do through scripting on a shared hosting account. I noticed that zen-cart has an option for gzipping. I cannot seem to find many resources about "how" to gzip. thanks in advance

Hi, Jason. Typically gzip is enabled through your web server configuration. That's the way I do it on my shared web host (Dreamhost). You can also do it via PHP, and even manually. This article by Stoyan Stefanov has more tips about the performance rules on shared hosting sites:
http://www.sitepoint.com/article/web-site-optimization-steps

-Steve

Hello all,
I think there is a problem with HTTP 304 response, which is reported as not gziped, and without Expire header even if is served from local cache and inital resposes are gziped.
Is any way to set Yslow to ignore 304 responses from server, until local broser cache issue is resolved?
Thnaks in advance

I just ran across the issue of YSlow not showing my files as gzipped yet http://www.whatsmyip.org/mod_gzip_test/ reported every url I sent it as being GZipped -- then it occurred to me - disable my Firewall.

Most Firewalls (at least Agnitum Outpost 2008 does) these days will inspect all of your HTTP traffic these days, thus busting the gzipped check.

As a suggestion: give two grade reports. one for external domain speeds and one for internal (on server) data. This way developers can have an easier time of maximizing their own server and then decide what they want to do about the external content. Perhaps change them to warnings or notices. But overall when I look at YSlow I want to see the grade of the content that I can directly control.

Thanks for the great addon!

May help some people (May not be correct!)

Getting css html text and js gzipped in apache2 add the line below to your .htaccess

AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css text/x-js

TOPTIP
Either clear your browser cache or disable it (eg with FF webdeveloper toolbar) otherwise yslow may report its not gzipped.

Any reason, when going to a load balanced url the gzip doesn't compress all components, but going directly to the servers the components are compressed fully?

Can .FLV file be gziped?

These rules seem to conflict.

The next rule says to put stylesheets at the top of the page for progressive rendering. Progressive rending doesn't work if gzip is enabled.

I believe I found an anomaly in YSlow. some extensions values in the yslow.js are not being being picked up. I have found the extension values are being defaulted to the values in lint.js, specifically:
pref("extensions.firebug.yslow.pointsGzip", 1);
pref("extensions.firebug.yslow.gzipFileSize", 700);
What I found was the yslowFirebug.js getPref() would receive a "undefined" type as the nsiPrefBranch type is INVALID (0)

Have you guys ever quantified the impact on the client side when decompressing GZIP'ed documents?

So assuming you've got a file being sent with compression, and in general you want the file to be cached also; given the issues with IE6 and below, do you also have to send a vary header on user agent, to stop ISP / proxy caches sending the data to IE versions that can't handle it.

My host (Hostmonster.com) has mod_gzip and mod_deflate disabled.

My website runs on Drupal 5.7

What can I do to serve GZiped html css and js??? Is there any workaround whn host is disabling mod_gzip and mod_deflate??

So far I *manually* Gzip .js and .css files and tell .htaccess to preference the compressed files instead of the the originals. Performance is definitely improved (though html files are still not being served a Gzipped), but Gzipping files every day can drive anyone insane. Is there a workaround?

Thanks

Just to echo Garen's comment, I would also like to know what can be if your host has disabled mod_gzip and mod_deflate?
I don't mind gzipping the javascript and css manually as I don't plan on changing these for a while to be honest :)
Are the PHP function(s) that should be used on this page of the manual http://uk.php.net/manual/en/ref.zlib.php ?

Nick Holmes brings up a good point: IE6 can't handle the Vary header. That would seem to prevent any attempt at proxy and CDN safe compression.

Out of all the performance tips, this tip rellay improved the performance of our application. Almost 3-5 times. Also for people who are worried about the CPU time. I would say it is easy to add another CPU to increase the processing speed. It is really hard to increse the Network bandwidth and also it is not in one user/group's control in the internet world.

Hi,

I was trying out some of these best practices.. looks like If I enable the apache gzip, I cannot use the "flush the buffer early" best practice.

If I insert a sleep(5) before the final output (which sends some html for div popups to the browser), the page loads and shows up immediately when i have not enabled gzip. 5 seconds later, the divs would load in the background.

When I enable apache gzip, the page would not show up for 5 seconds.

What do you think?

Hi,

I am using gZip compression for compressing JS and CSS. I have written this piece of code in my .htaccess file.

# Use PHP5 Single php.ini as default
AddHandler application/x-httpd-php5s .php

# Or, compress certain file types by extension:

SetOutputFilter DEFLATE

SetOutputFilter DEFLATE

SetOutputFilter DEFLATE

But it is not compressing JS or CSS. Please help me

Thanks

Thanks Steve for your "High Performance Web Sites" book. It's brilliant, and has helped me no end.

I have, however, come across an issue with regards to gzipping content that isn't mentioned in your book, or on this blog. In fact, I'm surprised no-one's really mentioned it because it was so obviously bad that I couldn't understand why I was the only one who seemed to notice this behaviour! Hopefully this will help other people out there who try and use this tip.

Basically, IE6 and IE7 don't handle the Vary header well at all. In fact, they're terrible with it.

When I implemented the gzip technique with the removal of ETags, adding Vary headers caused IE to reload every single image on every page view. Even leaving in ETags caused odd behaviour (e.g. ignoring far future expires headers and having 304 responses occur on a few files). This, of course, defeated the whole point of using gzip with far future expires headers.

If you're in the same position as I am whereby you can afford the bandwidth and ignore proxies, I recommend you add the following to the config for your site (htaccess or wherever you're defining your settings):

# set cache control to private to ignore proxies and avoid edge cases
Header append Cache-Control "private"
# Remove the vary header for MSIE
BrowserMatch "MSIE" force-no-vary

As soon as I did this, it worked perfectly as described.

What about SSL encrypted content? Is it worth gzip encoding content served from https URLs? From my limited reading, it looks like compression can be negotiated in the SSL handshake, but I don't know what current browsers/servers actually do. Looks like I'll have to do some experiments with a packet sniffer and report back...

Have done some investigation with WireShark. Making a single download of a 30568 byte JavaScript file from a https URL, counting the total length of all "encrypted application data" blocks in the SSLv3 or TLSv1 traffic.

Without mod_deflate:
Firefox 3.0.5 SSLv3: 31024 bytes
IE7 TLSv1: 30986 bytes

With mod_deflate:
Firefox 3.0.5 SSLv3: 10672 bytes
IE7 TLSv1: 10624 bytes

Looks like it is worth enabling mod_deflate if you are using mod_ssl.

As noted by "daphne", looks like progressive rending doesn't work if gzip is enabled and causes page to flicker in FF if I refresh, is there any suggestion/Fix to avoid this and take advantage of both? I tried gzipping only JS files by setting AddOutputFilterByType directives in Apache 2.0 but apparently that did not work neither.

@Andy:

> Akamai will not cache the content since it would need to keep two copies, one compressed and one uncompressed

That's incorrect. I've double checked Akamai's documentation and tested it as well, Akamai can be configured to honor the compression if the browser accepts it, or it can be configured to follow whatever the origin server does (e.g. serve it always uncompressed).

For example if look at the headers on this:
http://www.akamai.com/js/jquery-1.2.1.js, you should see Content-Encoding: gzip (as long as your browser's request contains: Accept-Encoding: gzip).

Why most (i guess all 99%) windows web hosting servers dont allow gZip compression.

Thanks
Heatl Patel

Sometimes, when an HTTP server returns compressed content, client based tools show that the client is receiving un-compressed content instead.

In particular, the server sends the response with header
"Content-Encoding: gzip"
but on the client, we see instead
"----------------: ----".

Wireshark shows the HTTP response reaches the client network interface with the Content-Encoding header intact.

In the two cases I've studied, it turned out the server's IP address was not trusted by the client firewall. Perhaps, for untrusted HTTP content that the firewall considers potentially dangerous, such as text/html and application/x-javascript, the firewall decompresses the response in order to inspect it, and then substitutes the decompressed content to save the client the trouble of decompressing again, replacing the obsolete heading with hyphens.

That the client firewall is a factor can be demonstrated by turning off the firewall, or by configuring the firewall to trust the server.

Ylow got "Compress components with gzip" in "304 Not Modified" HTTP code request.

But how gzip a request in within "304 Not Modified" HTTP code?

It's no felt/one-way!!

HTTP 1.1 deflate is zlib (delfate + headers)
HTTP 1.1 gzip is deflate + headers

Fact: GZIP headers are larger than zlib headers.

Internet Explorer thinks deflate means "deflate WITHOUT any headers" which is incorrect. This is why HTTP 1.1 deflate (zlib) is considered "less effective." HOWEVER, if you always send _deflate without headers_ all modern browsers can still successfully decode the data (iphone not tested).

Fact: Deflate, by itself, is smaller than gzip or zlib (when the same levels of compression are used).

More information:
http://stackoverflow.com/questions/1574168/gzip-vs-deflate-zlib-revistited

Yahoo! may be steering developers/technicians/engineers in the wrong direction on this one. Please fix.

I tested a site running on a Apache 2 Sever using Deflate. But YSlow says: "There are 11 plain text components that should be sent compressed". I tested it with another online tool and this detected Deflate.

Does YSlow can't detect Deflate or where could be an error?

Thanx

Frank