#SAM2013 Ribbon
Moving past page load time
Posted in: Community, Monitoring, Performance Testing, Real User Measurement   -   September 11, 2013

Thinking past Page Load Time

In the past, the performance of a website was measured by checking how long it took for an event called ‘window.onload’ to trigger, this measurement was dubbed the “Page Load Time” (PLT) of a website. Five to ten years ago, this was a very good indicator, but websites have evolved since then, so must our mindset when it comes to performance. Most of them have widgets and functionality that is loaded from other servers (social sharing, commenting systems, ads), combine that with more images and things like infinite scroll, the time it takes for the window.onload() event to trigger can vary. We need to start measuring the time in which a user can start to interact with the website. Tools like Webpagetest.org (WPT) have started to offer things like “Speed Index” and “Above the Fold Render Time”, but I’ve noticed very few people in the WordPress ecosystem aware of this new mindset.

The Current Mindset

Judging by tickets, sales inquiries, and forum posts, it’s clear the average developer or owner of a website is making ill-informed decisions, the pattern I see most often is as follows:

  1. Test website from www.webpagetest.org
  2. Focus on page load time, but ignore things like Speed Index, Render Time, Page Size
  3. Make the recommended changes like: add CDN, set expiry headers, combine javascript, etc
  4. Repeat step 1 , not notice any progress (page size on first render still hasn’t been reduced)

There are two problems with this approach, the first is the focus on Page Load Time without regard to page size, and the second has to do with the notion of making expensive decisions without collecting sufficient data.
WPT is a very good resource to understand the breakdown of your website and to identify basic issues with it, but it’s not indicative of actual traffic patterns to your website. In my opinion tools like WPT and Pingdom Tools are good for the following

  • Start Render and Speed Index
  • Total payload (size of page)
  • Understand breakdown of payload (css, images, html, js etc)
  • Breakdown of payload by domain (how much of the page is loaded from facebook,twitter as opposed to own domain)

The evidence against Page Load Time

Page Load Time (PLT) is a metric of the past. The tools that measure page load time have been updated to measure and report the performance of a modern website. While PLT is one of the metrics that is reported, it is not relevant for modern websites. If you are still focusing on it, or telling your customers to focus on it, you’re wasting time. Here’s what some of the pioneers and experts in the field are saying.

“Consequently, page load time, which has been the de facto metric of the web performance world, is also an increasingly insufficient performance benchmark – we are no longer building pages, we are building dynamic and interactive web applications. Instead of, or in addition to, measuring the time to load each and every resource (PLT), we are now interested in answering application specific questions” – Ilya Grigorik

Ten years ago, window.onload was a good proxy for the user’s perception of when the page was ready. Back then, pages were mostly HTML and images. JavaScript, CSS, DHTML, and Ajax were less common, as were the delays and blocked rendering they introduce. It wasn’t perfect, but window.onload was close enough.
Steve Souders

Turns out, Jared Spool and his team at User Interface Engineering had already figured this out in 2001, when they wrote the following paper “The Truth About Download Time”.

“There was still another surprising finding from our study: a strong correlation between perceived download time and whether users successfully completed their tasks on a site. There was, however, no correlation between actual download time and task success, causing us to discard our original hypothesis. It seems that, when people accomplish what they set out to do on a site, they perceive that site to be fast.

When we looked at the actual download speeds of the sites we tested, we found that there was no correlation between these and the perceived speeds reported by our users. About.com, rated slowest by our users, was actually the fastest site (average: 8 seconds). Amazon.com, rated as one of the fastest sites by users, was really the slowest (average: 36 seconds).”

If not Page Load Time, then what?

In my opinion, When it comes to website performance we should focus on how quickly a visitor is able to start interacting with the website. The faster they can start interacting the better.

Unfortunately, there is no agreed upon name for this in the web performance community. It’s been referred to as “Render Time”, “Speed Index”, “Critical Rendering Path”, and since the tools available to us are currently not capable of measuring/detecting this event accurately, we need to get innovative.

Since April of 2012, webpagetest.org has provided testers with two metrics that can help.

Making better informed decisions

These numbers provide a basic idea of how quickly our page may render, but as I said above we shouldn’t commit to major overhauls based on these tests alone. Fortunately, tools are also available that let us monitor what our actual customers/visitors are experiencing.

At ZippyKid, we utilize Real User Monitoring aka RUM, we encourage our customers to do the same. Implementing RUM is very easy, and there are services that fit all budgets and functionality.

What is RUM? And how do we use it?

Think of it as a service that fires off WPT, or Pingdom Tools every time your website gets a pageview. It stores the results in it’s internal database, which you can then query for some interesting reports. Since these reports are based on actual user visits, you get a better idea of how your website is being experienced. The reports can range from basic distribution of page load time, render time (mPulse), to breakdown by platform (desktop, phone, tablet) and state (network, server, browser).

Load Time Distribution with RUM

The following report from Pingdom tells us how the load time was distributed amongst visitors to our website over the past 7 days. We can see that the majority of our visitors were able to load the website in less than five seconds. Based on our business metrics, and performance of our funnel, our time and money may be better spent on something besides performance.

load time distribution from Pingdom

This report is from mPulse by Soasta, I couldn’t figure out how to give me this report for the same time period as Pingdom, but it’s a daily report.

load time distribution from Pingdom

What about Mobile?

Here are some guidelines by Google, that should help you optimize your website for smartphones and tablets.

What should we do for SEO?

This article from the Google Webmaster Central blog, has caused a stir amongst the SEO community. But, they seem to be ignoring the part where Google says speed is one of two hundred factors that Google uses to rank a website, it’s safe to assume that uptime, and relevance of the data on your website is more important. If your website ranks on page 5, reducing page load time alone will not move it to page one.

“While site speed is a new signal, it doesn’t carry as much weight as the relevance of a page. Currently, fewer than 1% of search queries are affected by the site speed signal in our implementation and the signal for site speed only applies for visitors searching in English on Google.com at this point.”

I am not aware of anything else from Google that has shown a change in policy. Even the latest Penguin update from Matt Cutts does not mention this.
If you’re worried about SEO follow the quality guidelines provided by Google

Looking Forward

I would like to see the discussion regarding website performance move past page load time, towards Render Time and Speed Index. I hope this post was helpful, and can form the basis of a transition towards websites that are truly optimized for user experience.

Note: This post took three weeks to draft, it would not be possible without the editing help of my co-workers, the research and feedback of people like Steve Souders , Ilya Grigorik , and Jared Spool. Thanks to all of them for all their time and insight.


Vid Luther is the CEO and Founder of ZippyKid, the most reliable WordPress Hosting platform for businesses. For more tips, tricks and myth busting information, head over to the ZippyKid blog.

  • http://blog.justindorfman.com jdorfman

    Great post Vid! Now I know why it took so long to get it out. =)

    • http://zippykid.com/ Vid Luther

      For a post on speed, it took a long time to write :). Thanks for giving me a forum.

  • http://omaruddin.com/ Omar Uddin

    Awesome post Vid!

  • Randy Bear

    Great post and love the reference to some of the tools. Have already found problems on my own site. External plugins are killers, sometimes.

  • Mike Schoeffler

    We’re still in early days – need some real studies to back up intuition about better measures (probably best is close to Above The Fold Render Time, but needs a catchier name :) ).

    We also need wider distribution. The biggest RUM package of all (Google Analytics) doesn’t contain anything past time (by default).

    Good post – we’ll get there by thinking through all these issues.

    • http://zippykid.com/ Vid Luther

      Thanks Mike,
      What kind of industries are you guys doing your tests in? What can I or others reading this post do to help distribution? Or should we stay put and be patient? :)

      • Mike Schoeffler

        Kind offer, but I wrote unclearly. Too many pronouns :).

        Rephrase: “the speed industry is in early days of understanding the best speed measures and also needs wider distribution of various speed measures”.

        My work at workglide.com statistically infers the profit impact of a website’s speed improvements across different pages – before investment in heavier engineering. Helps add a bunch of money to the bottom line that’s directly attributable to developers.

        Important, but not adding any new speed measures. The work applies to large sites in any industry – if this can help you out, just drop me a line – mschoeffler at workglide.com.

        That said, we’ll eventually have the data to objectively evaluate different speed measures and will post whatever we figure out. I expect the winner to be something like Above The Fold Render Time, but data’s better than intuition.

  • Brad Canham

    Great post Vid!

  • http://www.blueprintmarketing.com/ Thomas Zickell

    Vid great post very informative. when Matt Cutts states that speed is part of the algorithm I think this is important even if it is 1%. Very few sites have effective SEO may be 2% more likely 1% however speed affects Google in different ways if you have a slow website and people hit the back button after selecting your website for a query Google will assume that person has not found what they are looking for therefore if this continues Google will change your rank based on user actions. called “Pogo sticking” in SEO. Another important thing speed adds is a good user experience. Everyone here has left the site because it is just too slow. Well if your site is just too slow and your click through rate goes down along with the amount of time people spend on your site in between clicks Google will make assumptions wisely on user actions and penalize slower sites.

    I do believe that the TTFB that Google currently admits it is using to measure speed is out of date and not a true test of site speed. Therefore Google gives such a small amount of power to load time. However this should not tell people it is okay to use junk hosting a.k.a. anything under $5 shared. I strongly recommend using a managed WordPress host like ZippyKid or WP Engine for WordPress and an enterprise host Such as FireHost or Terremark for anything else. just my $.02

    PS how do you feel about Neustar’s RUM tool’s or NewRelic?

    Excellent post.
    Respectfully,
    Thomas

  • http://www.tonilaird.com/ Toni Laird

    Very informative post Vid.