Tackling the basics of web analytics: Getting the right numbers right

This article, written by Neil Mason, was originally published on Clickz.com and is republished here with permission.

It was one of those moments. I was working on a client’s data and I began to suspect that something was wrong. Not with the client’s business but with the data, but the potential business implications were very significant. Sure enough, as I dug deeper and deeper into the issue it became evident that there was something seriously wrong and I got that sinking feeling.

The story behind the story was that the business in questions was looking to aggressively improve the effectiveness of the digital channel and had been focussing on conversion optimisation as traffic levels were quite buoyant. They had implemented a satisfaction tracking survey to understand visitor intent and satisfaction; they had commissioned usability testing to understand the user experience in more detail and they had started a testing and experimentation programme. It was all the right stuff, the problem was that there wasn’t any strong evidence that conversion was actually improving. So it needed a deeper dive into the data to find out what was happening and that’s when the problem emerged. Without going into the gruesome details the impact was that the conversion rate was being underestimated and that the degree of underestimation had been getting worse over time. This meant it was a case of “What do you want first, the good news or the bad news?”. The bad news was that the historical data from the web analytics tool was wrong on some key metrics, the good news was that the conversion rate was better than previously thought.

The really bad news was actually that the business had potentially been focussing on the wrong problem. Whilst all the activity on conversion optimisation was good stuff, the revised data highlighted that other issues may have been more pressing. The other bad news was that the credibility of the data was seriously undermined and to some extent the team as well. For conversion optimisation it was taking two steps forward and one or two steps back.

The point of this case study is that it reinforces the need to get the right numbers right and to keeping them right. When it comes to marketing optimisation good quality data is a core component. Getting good quality data that allows better decision making is a key step on the journey. That might seem like an obvious statement but it is not a process that should be underestimated and nor it is a one off set up event. When a new system is implemented that is inevitably a focus on the data it is generating and that might be reconciled against other data sources. That’s great but those checking processes need to be repeated at regular processes to ensure that the data integrity remains high. If this isn’t a managed, ongoing process then there is the possibility that the integrity will decline over time until something happens which causes the data to be questioned by which time it might be too late.

Managing data integrity is a messy job but someone has to do it. Good processes will certainly help ensure that all pages get tagged; campaigns are tracked properly and so on. Technology can also help with solutions out there that will check for tags on the site, as well as solutions that help address tag management challenges. It also needs a keen eye to be looking at the data for trends and patterns that may not be a true reflection of what’s going on. I actually think this is a skill but it’s a skill that can be learned. A good marketing analysts can sense when something doesn’t look right and in my own experience if something looks odd , then it probably is odd and isn’t real behaviour. Sudden changes in trends, steps in the data, spikes and dips are all potentially symptomatic of artificial impacts on data and if they cannot be explained by real world events, then it’s worth digging into the data to see if there is anything untoward happening like changes to the tool’s configuration, new site monitoring tools being put in place, changes to the hosting environment and so on.

So, don’t see getting good data integrity as a one off event but as an ongoing process. Be wary of the potential impact that changes to your site or your tracking environment will have on your data and plan accordingly. Take time to reconcile your data on a regular basis to see if there are any divergent trends. Hopefully with these basic processes in place you can avoid that sinking feeling at some point in the future.