Caching everything – step by step
Posted in: Getting Started, Speed Up Tips   -   August 29, 2013

Over the last five years we’ve seen a surge in web traffic. As you have surely read a hundred times before, handling the traffic in a timely manner is very important. Of the many tools you’ll need in your war-chest a caching reverse proxy is imperative to battle the ever increasing traffic. Caching reverse proxies are important because they are fast. Most of the time they take a piece of content stored in memory and push it onto the networking card. Compared to running a complex environment such as PHP, Python or Java and having that environment generate the content a reverse proxy requires just a tiny fraction of the resources. A caching reverse proxy would require no more than 1/1000 of the resources typically required. As you all know this is good news for scalability and UX.

Deploying a caching reverse proxy can be a quick and easy task, and the gains depend on how much effort you put into it. Below I’ll walk you through the most common deployment process. We’ll be talking about Varnish here, because that is what we know and love, but most of the steps could be done with other reverse proxies as well.

Step 1 – Caching static contents

Caching static contents is a simple way to start using a reverse proxy. As the tricky thing with caching is mostly about cache invalidation you avoid that whole problem by only caching content that doesn’t change. That would be content such as CSS, static HTML, javascript, images and similar.

The gain is small, but you reduce the concurrency and possibly the CPU- and IO load on your backends. Since a reverse proxy doesn’t have to interact with the filesystem in order to deliver content a reverse proxy can potentially be faster. However, the gains here might be just as big by using one of the high performance web servers.

Step 2 – Adding compression

Varnish, as most other reverse proxies can do compression of content before it is delivered. Even a minified version of JQuery can be shrunk from 91KB to 32KB. Having compression enabled in the proxy layer can sometimes be simpler to set up than enabling compression in the plethora of various delivery systems that might be behind the caches. Also, since Varnish doesn’t use much CPU, you can shift some of the CPU load from your application servers.

The risk and effort of adding compression are low but the payoff are typically big. As a bonus if you do cache the object it won’t have to be recompressed the next time it is delivered, saving some additional CPU and latency as compared to compressing content in a load balancer or simple proxy.

Step 3 – Caching semi static content

The origin server might render static pages dynamically. This typically happens without regard to whether the page is changed or not. Most pages change infrequently and we can save a huge amount of resources and reduce loading time significantly by caching these. Even if we only cache them for a short period of time, say 30 seconds, this can completely remove all ill effects from a otherwise devastating traffic peak.

This is still pretty simple stuff to implement but the payoff can be significant.

Step 4 – Active cache invalidation

The end goal of a Varnish deployment is to have the origin generate each unique piece of content only once. In order to do so you need to have a mechanism to let Varnish know that the content that it keeps in its cache is updated and needs to be fetched again from the origin. This way you can have Varnish keep the content in memory forever and not a single CPU cycle will be thrown away.

This is of course easier said than done and we’re now at a stage where we need to do changes to the origin server. It needs to notify Varnish that content is changed. Typically this happens through a HTTP PURGE call against the unique URL of the page that needs updating. On simpler sites this is relatively straightforward. On more complex sites where dynamic objects are referenced on multiple pages this can be somewhat more challenging. To solve that issue in Varnish we have deployed some advanced cache invalidation strategies. The benefit of implementing these can be truly significant.

Step 5 – Doing content composition in Varnish

Some sites, media and e-commerce sites in particular, have content that consists of elements that are shared between pages. A good example of such content are lists of the 10 latests SKUs or the 10 latest articles. Since these lists are on every single page updating the lists will essentially throw out every page from cache. This can wreck total havoc on your hit rate and consequently your backend performance and the UX.

The solution is to use Edge Side Includes. It is a content composition language for HTML where you can load sub pages into other pages. Varnish will then stitch them together when delivering them. Varnish will also keep a parsed version of the page in memory and delivery is only a matter of pumping out the relevant memory regions.

If you use this Varnish will only keep one copy of each list. And updating that list only requires it to be fetched once and then all your pages will immediately be updated.

The payoff is high, but as you probably understand there is some work that needs to be done on your origin.

I should note that you can achieve more or less the same by doing content composition on the client side. Personally, I like the idea of the content being complete when it leaves my network, but that might be because I’m a backend sort of person. Also I suspect that performance might suffer a bit as you’ll add more latency to the content composition on the client side.

What about real time content?

We talk to several people that are concerned that caching will result in stale content being delivered. They are of the belief that web caching is bad for real time content. However, you need to keep in mind that even a second of caching will result in dramatically reduced load on your backend. If a hot object is requested 100 times a second adding a second of caching means your backend only has to fulfill 1% of those requests. And it is very doubtful that your users will notice or care about that delay. In some cases we’ve even deployed caching of 0.5 seconds or less. On systems dealing with huge amounts of traffic this can be a determining factor when it comes to staying up or going down.

Summing up

We are frequently approached by websites that struggle. We usually manage to get them into fairly good shape in a matter of a couple of days. As with most other technologies the easy parts are easy but the hard parts can be really hard to implement. The payoff however of putting in the effort of properly implementing reverse proxy caching can be significant. We’ve seen our customers experience dramatic increases in traffic, user satisfaction and conversion levels due to the increased performance.

Good luck with deploying your Varnish chosen caching proxy! You will not regret the effort.

Tags: , , , , , , ,