« MyBlogLog API documentation available | Main | Mashup Zimbra Using Zimlets »
February 4, 2008
Performance is important to users. It influences click through rates, loyalty, and engagement. Users want fast websites that they can view from anywhere, including phones, wireless connections, laptops, and home computers. Users also want a graphically rich user experience with all the bells and whistles. Multimedia blends of video, images, feeds, and other components can be very slow indeed. Here at Yahoo! we spend a lot of time analyzing and improving the performance of our own sites. In honor of Super Tuesday, we thought it would be fun to take a poke at the Presidential Candidates web sites and share with you what we found.
How did they do? Overall, atrociously, all the candidates failed the YSlow exam except Mike Gravel who earned a "D". Page weight was a problem for Barack Obama, whose site weighed in at almost 700Kb. It was even worse for Mitt Romney, whose site weighed a whopping 1,531Kb. I hope he doesn't have supporters trying to make contributions on dialup modems!
Democrats got better grades in almost all performance subjects tested, in particular response times and page weight. They improved user experience for returning visitors by setting an Expires headers and improving the full cache user experience. This helped propel them to a performance GPA of "C" despite their failing YSlow grade. Republicans never managed to overcome the deficit and finished the semester with an "F".
Figure 1: Democrats versus Republicans Performance Report Card
Performance is one component in a balance of competing goals. These sites are trying to solicit support, donations, and volunteers. They need to provide a rich user experience that keeps people coming back, and engages them with the candidates' progress. Ultimately, they want to do this in a way that is fast, and accessible to as many voters as possible; including those on mobile phones, dialup modems, or low broadband.
Mike Gravel and Hillary Clinton's websites had the two best response times tested. Barack Obama's website came in fourth, after the leading Republican. Response time is all about getting the biggest user-experience bang for your buck. One reason Obama's site might be slower, is the amount of below-the-fold content. Many voters may not see this extra content, we human beings don't seem to like to scroll, but it still impacts performance.
Clinton, Obama, and Huckabee have graphically rich sites, and yet they are among the fastest. Do these sites succeed in engaging voters, or do they prefer the more serious, austere look of Gravel's site? The idea that a candidates website has a serious impact on his or her chances of being elected is relatively new. The rules and strategies are being invented now; we're living history.
Figure 2: Below the fold content might not be visible to voters, but it does affect response times
Figure 3: Democrats Response Time Report Card.
This graph shows all of the candidates' grades plotted together. Reaching the outside band means the candidate got an A, for instance Obama and Gravel received As in Image Optimization, while Clinton squeaked into a high-B. The closer the candidate got to reaching the edge of the ocatgon, the better their grade. In fact each band is equivalent to one letter grade. From roughly the halfway point to the very center are variations on a failing grade.
Clinton and Gravel provided two of the best performance-based user experiences recorded. However, Clinton had a large number of HTTP requests, which can slow down a site significantly. She counteracted that by setting an Expires headers so that returning voters would not have to pay the same performance penalty. Clinton also split static content across more than one domain to enable parallel downloads.
Examining Republicans performance offers a clear opportunity to witness the connection between response time and page weight. To deliver a fast site, hard choices have to be made about which features to include, and which to abandon.
Mike Huckabee's website combined low page weight and fewer http requests to achieve the best response time among Republicans and the third fastest response time overall. He could however trim 20Kb of fat from images with no loss of quality. We tested using lossless compression algorithms to determine how much extra baggage the candidates' pages were carrying. Romney was the real surprise in this category. The extra fat in his home page weighed more than Mike Huckabees entire page!
Image Optimization is the kind of low hanging fruit that makes your site faster with absolutely no loss for the user. The tool we built to test image formats and compression algorithms determined that John McCain was the master of image optimization; we were only able to remove 1Kb from his images.
Mitt Romneys site takes almost eight seconds to load even on a broadband connection. He got so many "F"s, our radar chart looks like Pollock on a bad day.
Figure 4: Republicans Report Card
Every one of these sites had outlying data points, that is, random response times of as much as 19 seconds. Romney even had a data point at 1.825 seconds, despite his more typical 5-10 second load times. These bad user experiences are real, and while most users just press reload and forget about it, it is important that we correct what we can, before they get frustrated and simply don't come back.
Comparing worst-case scenarios, medians, averages, YSlow scores, empty or primed cache experiences, and other measurements can help you get a fuller picture of your user experience. Rather than looking for one magic number, performance requires us to unravel a nuanced puzzle. Try to understand what your users see, and how you can make it better for them, whether they are voters, customers, or people coming to read your blog.
For those of you who like numbers, here's the data:
| Hillary Clinton | Barack Obama | Mike Gravel | Mike Huckabee | John McCain | Ron Paul | Mitt Romney | |
| YSlow | 59 | 52 | 66 | 50 | 42 | 36 | 33 |
| Page Weight (K) | 300 | 691 | 195 | 185 | 568 | 483 | 1531 |
| Response Time (S) | 2.6077 | 3.4810 | 2.3733 | 3.1148 | 6.5309 | 4.2125 | 7.5352 |
| HTTP Requests | 96 | 94 | 32 | 56 | 107 | 83 | 77 |
| Cookie Weight | 214 | 333 | 281 | 217 | 299 | 203 | 130 |
| Primed Cache - Page Weight (Kb) | 33 | 242 | 8 | 46 | 237 | 173 | 448 |
| Primed Cache - HTTP Requests | 4 | 93 | 4 | 56 | 107 | 76 | 77 |
| Wasted Image Weight (Kb) | 29 | 6 | 20 | 20 | 1 | 49 | 205 |
All measurements were taken using a MacBook Pro with Firefox, Firebug, and YSlow over a wireless connection. Page weights, cookie weights, and HTTP requests were determined via the YSlow Stats panel.
Should you choose the next president based on their YSlow score? Probably not, but, it is one of many interesting ways of evaluating how technically savvy they are in an increasingly technical world. Now don't get me wrong, I know the candidates didn't write their own HTML or optimize their own images, but they did choose the person who would do this work for them. That's what makes this game interesting; they have to choose the right people for the right jobs every day. Happy voting!
Nicole Sullivan, Technical Evangelist
Yahoo!'s Exceptional Performance
Posted at February 4, 2008 9:09 PM
I enjoyed reading your excellent report cards on the candidates.........lets see if the voters feel the same way
Posted by: Vectorpedia(Rick) at February 5, 2008 3:59 AM
I just tried out the one that was said to be the worst offender and it loaded in about three seconds. It still had an F on YSlow though.
Posted by: Christian at February 5, 2008 9:37 AM
Christian: Individual response times are pretty variable, and all of the candidates had outlying data points. Generally speaking, the higher page weight sites had the most variation.
Posted by: Nicole Sullivan at February 5, 2008 3:28 PM
@Christian: Also keep in mind that Nicole's measurements is an average of page loads taken on a MacBook Pro with Firefox over a wireless connection in one geographic location. Generally, response times for users vary depending on multiple factors including bandwidth, browser type, geographic location, ISP, computer specifications, etc.
Posted by: Tenni Theurer at February 5, 2008 5:18 PM
Got it!
Posted by: Christian at February 6, 2008 10:58 AM
Clever idea
Posted by: Wesley Tanaka at February 7, 2008 12:35 AM
Comment Policy: We encourage comments and look forward to hearing from you. Please note that Yahoo! may, in our sole discretion, remove comments if they are off topic, inappropriate, or otherwise violate our Terms of Service.
Copyright © 2008 Yahoo! Inc. All rights reserved.
Privacy Policy - Terms of Service - Copyright Policy - Job Openings