When is a visitor not a visitor?

This article, written by Neil Mason, was originally published on Clickz.com and is republished here with permission.

When is a visitor not a visitor? The answer is… “when it’s a cookie”.
The controversy surrounding one of the core metrics of web analytics, Unique Visitors, has raised its head again. In a recent blog post Eric Peterson, the web analytics consultant and author, called on the web analytics industry to stop using the term Unique Visitors because it doesn’t accurately reflect what is actually being measured. The issue stems from a set of proposed definitions from the IAB on audience reach metrics, including Unique Visitors. The definition of Unique Visitors from the IAB is different to the definition commonly used in the web analytics industry which is cookie based.

I agree with Peterson’s sentiment (if not entirely with his style and approach) as I have been saying for a long time that the Unique Visitors metric from a web analytics tool is potentially very misleading. I have spent a lot of time in workshops and client engagements over the years explaining what is actually being measured by a web analytics tool when it reports on Unique Visitors. It is fundamentally different to the way that the Unique Visitor metric is defined in an audience panel such as Comscore and Nielsen NetRatings. For a lot of people this is highly confusing. For publishers and other sites that depend on being able to demonstrate the size of their audiences, it’s a bit of a nightmare.

Web analytics systems define a “unique visitor” based the presence of a cookie. If I visit a site using three different devices in a week (say a PC, a laptop and a mobile phone) I will be recorded as three different “unique visitors”. If I also regularly delete the cookie then I can appear to be a new visitor to the site and I’m therefore not counted as being “unique”. Audience measurement panels define a Unique Visitor based on the activity of an individual member of the panel. Web analytics tools measure all activity on a website (a so called census-based approach) whereas a panel measures a proportion of the activity on a site (a sampling approach) and then uses that to estimate the total.

So with two different definitions and two different data collection methodologies, it’s hardly surprising that people can get confused and debates rage about which numbers are right and which numbers are wrong. It would certainly be helpful if the Unique Visitor metric was called something else in a web analytics tool and that’s a massive issue for the industry to address, hopefully in a collaborative manner but I don’t think it’s an issue that’s going to get solved any time soon. For site centric measurement, cookie-based tracking is the closest that we get to the notion of a visitor. There are some specific instances in which it becomes possible to link cookies to real people to get a proper unique visitor count, but it generally involves massive amounts of processing and massive amounts of sensitivity to privacy issues.

However, what this debate reinforces for me is the need for “information consumers” to understand how any of the data they use for business decision making is put together. That means it’s important that the web analytics industry continues to educate users of this data on its “provenance”, warts and all. One of my biggest concerns about the Unique Visitor metric from a web analytics tool is not actually its “technical efficacy” but more the messages that it sends out. The implication from reporting on visitors from a web analytics tool is that you can fully understand your website visitors using clickstream behavioural data. Because we call this metric “Unique Visitors” it lulls us into a fall sense of security that we are actually tracking “visitors” rather than “devices” and therefore we don’t really need to understand anything else. However, clickstream data doesn’t tell you anything about who is visiting the site, why they are there and what they thought about the experience.

In my last column I talked about a simple maturity model. The final stage of this model was “customer centricity”. Customer centricity involves having a 360 degree view on your customers and being able to answer the “who” and “why” questions, as well as the “what” and the “when”. Web analytics is just one tool in the tool box. Absolutely necessary but rarely sufficient. So the Unique Visitor debate will go on. As I said I doubt it will be resolved any time soon but if the discussion reminds us of what we’re measuring and how we’re measuring it, then it’s a useful debate to have.

Disclosure: Neil is a Board member of the Web Analytics Association (WAA), the industry association concerned with setting standards and definitions of measurement in web analytics. The views expressed here are Neil’s personal views and are not, in any way, an official view of the WAA.