How Do You Measure Visitor Engagement?

What’s the most important metric when it comes to determining the health of your site? It’s not really a fair question since no one metric tells the entire story, but for a long time page views have been considered one of the most important metrics. Page views don’t tell the whole story of course. Does someone look at a lot of your pages because they like your content or because they can’t find what they’re looking for? And with the rise of Ajax applications how meaningful is to to measure when your entire application is on one page? Recently there’s been a movement toward time spent as a measure of engagement with your site, though that has it’s share of problems too. So how is one to measure visitor engagement with your site.

Yesterday I came across a post at Fresh Egg with a new formula called the engagement factor, which I think has a lot of merit in measuring how people interact with your site. The formula is pretty simple.

Engagement Factor = (Avg time on site (seconds) x Pages per visit) / Bounce Rate

What I like about the engagement factor is it makes sense. It measures page*seconds/visit which does sound like engagement. While time spent and pages per visit each aren’t perfect in measuring engagement, chances are as each value increases a visitor is interacting more with your site.

Again that’s not 100% true. Visitors might visit a lot of pages because they can’t find what they want. They may leave your page open in a tab in the background while they surf other sites and close your page before ever reading it. But combined they probably help to limit the other’s inaccuracies. The person who leaves the page open in the background and closes it isn’t visiting a lot of pages. The person who can’t find what they’re looking for doesn’t spend a lot of time on the site.

Not perfect, but better.

Considering bounce rate it makes sense that people who never click beyond the entry page aren’t engaging with your site. As bounce rate goes up engagement goes down. That’s exactly how the formula calls it too. Looking at the example from the Fresh Egg post it seems like they’re using 50 as the divisor in the equation when the bounce rate is 50%. It would be better to call it .50, but since the end result is simply a ratio and the number are more readable using 50 it’s not a big deal.

My post on developing a 2 column css layout has usually done well bringing search traffic. Two thing I’ve always noticed is the page has a high bounce rate, but visitors also average a fair amount of time on the page, more than enough to read and digest it. Now I’m probably not doing a good enough job getting people to click through from that page to another on the site, but the high average times on the page lead me to believe people are finding it useful. I would think they are still engaged with the page even if the page doesn’t lead them to visit the rest of the site.

Given the engagement factor is calling for pages/visit this isn’t something you can use to measure interaction with a single page, but rather your entire site. And it is something that will need to be looked at in relation not as an absolute. Saying your engagement factor is 25 is meaningless except in comparison to last month’s 22.

The engagement factor might not be perfect, but from what I’ve seen so far it’s as good as any metric for measuring how visitors interact with your site. I think it’s a good start toward measuring visitor engagement that could possibly be developed into something more.

What do you think? Does the engagement factor seem like a good way to measure user interaction with your site? What might be missing? What flaws might it have? Would you use it to gage the health of your site?

Download a free sample from my book, Design Fundamentals.

8 comments

  1. Forrest I was thinking about repeat visitors too. Before writing this post I checked my stats and compared the engagement factor (EF) for all visitors and repeat visitors. Repeats has an EF twice that of the overall. You could use EF to compare engagement from different search engines or for different keywords, etc. That’s part of why I like it.

    I think your example is a fair one, but keep in mind that EF can’t measure engagement for a single page so the one case may not play too much of a factor unless it was pulling in traffic not in line with the rest of your site.

    I’d still like to know how well I’m engaging new visitors since all repeat visitors were new at one point.

    I think the way to use EF would be to say your engagement has gone up in the last month after you made some design changes or that visitors from Ask are more engaged with your site than visitors from Yahoo!

    It’s an interesting ratio and one that maybe could stand further development. I think using both time and pages/visit is a very good starting point. I do think bounce rate is a good add to the equation, but perhaps there are different ways to include it or maybe other metrics that could be added to the ratio.

  2. I’m curious to see how the numbers compare with and without the bounce rate as a factor. Bounces are kind of a special case, and you’re right that they’re part of engagement, I’m more interested in how well I’m engaging the visitors I manage to keep…

    I have a photo on a page titled “Daddy’s Girl.” ( http://forrestcroce.com/Photos/Daddys-Little-Girl.html ) It’s a family portrait, but considering that guy on WMT who promised a link to anyone who could tell him why Google is dropping him, and turns out to have a lot of porn links … in hindsight, I can see how the title could be taken out of context. If people come in looking for fetish porn and find a G-rated, but beautifully captured photo, they’re going to bounce.

    In our example, that happened, so I couldn’t just sweep it under the rug Enron style. But I’m not interested in making my site appeal to the porn crowd, so in this case, the bounces are telling me more about server load than engagement.

    But that’s a convoluted example that’s on my mind because of a strange, unusual question I just answered. My design is my weakest link, and if I can improve that, I’m sure I’ll lower my bounce rate. So I’m going to track the “engagement factor” with and without bounce rates for a few months, and see if the trends make sense.

  3. Excellent points you all raise regarding the usage and limitations of the EF. By no means do I believe this equation to be flawless, and I welcome any and all suggestions for improving it.

    In order to track the EF of a single page I have been using the value of “1” as the pages/visit. Whilst this is almost redundant, it keeps the equation in tact and we are then still presented with easily comparable figures, which was my goal all along.

    I agree that the EF is better suited to comparing a single site month-to-month than it is for comparing different sites, but the same could be said of many of the common ratios we use throughout everyday analysis.

    Oh and well spotted that I’m using the Bounce Rate percentage as opposed to a ratio, do you think that is something I should clarfiy within my original post?

  4. Thanks for stopping by Spencer. I really like EF as a metric and I’m glad you developed it even if it’s not perfect. I guess we should thank Ammon too.

    That makes sense when looking at a single page. I was trying to think of other things you might use, but haven’t really come up with anything. My thoughts revolved around using page views for the page as some percentage of overall page views, but you wouldn’t really be comparing apples to apples any more. There might be something in there, though about engagement with a single page in relation to the engagement of the overall site.

    Using ‘1’ probably makes the most sense.

    I like how you can compare one set of traffic to another with EF. It was easy to see that my repeat visitors were more engaged with the site than first time visitors for example.

    I think it would be a good idea to clarify how you’re using bounce rate in the equation. MY first instinct was to turn the rate back into a decimal so 0.50 for the 50% bounce rate. It doesn’t matter one bit in the overall equation since either way you’re just dividing by a number and using the percentage does make the result more human readable.

    It might be good to explain that, though it’s not ard to figure out by seeing the results you got for your client. I was thinking one example calculation would be enough.

    I really like EF and I’ll see what I can come up with in an effort to try and improve it. Something tells me it doesn’t need all that much improvement though. The more I think about it the more I think it’s pretty good as is. It’s not perfect, but in part it’s because you really can’t measure visitor engagement all that accurately. There are so many variables the stats can’t show.

  5. Hi Steven,

    What an interesting concept. I might start using it with some of our clients.

    I was about to say that this metric is excellent for determining the health of a blog, not necessarily a regular site. But the more I thought about it, the less certain I am about saying so. The reason I bring this up is that I usually notice that the bounce rate for a blog is usually higher than the rest of the site.

    What do you think?

  6. It is interesting isn’t it. And while I’d like to claim the idea as my own let’s thanks Spencer a few comments up since this is his (with a little help from Ammon Jones) creation.

    I’ve noticed that too about this blog. Posts typically, though not always, have a higher bounce rate than the rest of the site. I think it’s because the posts attract a lot more long tail searches for queries that aren’t really relevant to the post.

    I think EF might be a good metric to look at the health of your site or blog. It’s a ratio and really needs to be looked at in comparison. I don’t think the absolute numbers will necessarily mean anything in the same sense that 1,000 visitors means something. But I think it can be something very useful in understanding if a change you make to the site leads to more engagement with the site or to see which of your visitors interact with the site more.

  7. Thank you also Steven for the link and excellent discussion provided here.

    I have update my original post with a clearer definition and a brief example as per your suggestion, although as you rightly point out for the purpose of considering one set of data against another, it doesn’t really matter whether you use a percentage or a ratio, so long as it’s the same equation used to each calculation.

    As will most metrics the EF is well-suited so sites that are being constantly worked on and (hopefully) improved. I use it for measuring both seasonal fluctuations (in order to gain an understanding of how seasons/times of year may skew change-related data) and also fluctuations based upon recent site changes. It is particularly good for blogs in that you can measure how well the blog itself (outside of the actual blog entry that the visitor landed on) actually captures the audience and convinces them to read further posts. Plugins such as “Related Posts” seem to have a very good effect indeed.

    I capture blog stats independently of the rest of the site in Google Analytics for as you mention, they are usually very different to regular site stats and require a different approach.

  8. Thank you Spencer. I like the brief example. It helps clarify how to use the formula. It doesn’t make any difference in the eventual comparison, but might as well have everyone using the same numbers.

    I agree it’s best to use EF to measure any improvements you make. That’s good to know about “related posts’ plugins since I’m planning to add them in a new design that’s hopefully coming soon.

    Blogs do seem to differ from the rest of the site as far as stats are concerned. Many of your loyal visitors won’t even read your blog on your site, which further increases bounce rates.

    Thanks again for developing the EF metric. I really like it and think it is a good way to measure engagement. I’ll be interested to see how you or someone else possibly adds to it and it’s something I’ll be thinking about as well.

Leave a Reply

Your email address will not be published.

css.php