KPIs

Fifteen or so years after organisations first started to measure what was happening on their websites by parsing log file data, what does good look like? There are five characteristics that define an analytically empowered organisation.

• Clearly defined key performance indicators (KPIs)
• Holistic approach to measurement
• Integrated data strategy
• Investment in “humanware”
• Ability to execute

Clearly Defined KPIs

At analytically empowered organisations, considerable effort goes into defining digital key performance indicators. To do this, the digital strategy must be clear and coherent. If the strategy isn’t clear, how can you possibly measure its success? In my experience, defining good, robust KPIs is not an easy task. As a result, KPIs are often not very good. Going through the process forces an organisation to think hard about its strategy, define what success looks like, and make a commitment to measurement. If you can’t measure it, then you can’t manage it.

Holistic Approach to Measurement

The old saying goes, “If the only tool you have is a hammer, then every problem looks like a nail.” Ever since the log file was developed, the digital marketing industry has been banging away with its web analytics hammer. The analytical empowered organisation understands that it needs a whole toolbox. Web analytics provides some but not all of the answers about digital performance measurement. It’s great for telling you what is going on but even a well-configured web analytics tool (itself a rarity) isn’t very diagnostic. Organisations need to invest in additional quantitative and qualitative data sources to truly understand what is going on and why. Additional investment requirements include voice-of-the-customer feedback on a number of levels (based on visit, page level, and post-experience), ongoing user experience measurement and analysis, site performance tracking, and contextual information about trends in the marketplace.

Integrated Data Strategy

A holistic approach to measurement also requires a unified approach to data integration. An organisation also needs to understand how all the pieces of the jigsaw fit together. This requires some effort around data definition (what the metrics actually mean) and where different types of data will be housed. In an ideal world, data is integrated around known users but this may not always be appropriate or possible. Different data types have different characteristics, so planning is needed to understand how the different components fit together. For example, some internal data may be on a customer level, but digital data is often based on cookie level data and one customer may use a number of different devices to interact with the organisation resulting is a number of different cookie records.

One powerful outcome of data integration is the ability to match behavioral data with data around attitudes and opinions. By integrating web analytics data with voice-of-the-customer data, it’s possible to look at the relationship between what people do in a digital channel and the experience they have. This type of integration gives the organisation the ability to measure outputs (things that happen in the channel) and to understand outcomes, which are often the most important things to know.

Investment in Humanware

All the hardware and software in the world will get you nowhere without “humanware” to extract insight and value from data. Too often in the past, investments have been made in technology without appropriate investments in people. The result is often disappointment, if not failure.

Today, analytics teams are taking a more multi-disciplinary approach. As data becomes more integrated, an integrated approach to analysis and insight is needed as well. Web analysts must start working alongside customer insight specialists and user experience experts, sharing their knowledge and expertise.

Ability to Execute

Organisations gain a competitive advantage from the application of insight, not by the generation of insight. Insight has no value unless something happens as a result. So the analytically empowered organisation has the ability to execute and make decisions. This has implications beyond the immediate concerns of analytical technologies; it also concerns a business’s entire technology landscape. Often, a product or site development process and technology constrain an organisation’s ability to affect change. So the analytically empowered organisation must develop strategies in technology and processes that enable it to act on its insights.

This article was originally published on ClickZ

“Why do you have a website?” I love that question; it tends to focus the mind. Any time you develop measurement frameworks for digital channels, you must be focused. Digital platforms, like websites, often have multiple stakeholders with different goals. There’s always a danger that overall objectives may not be clear. And if the objectives are not clear, then success is next to impossible to measure.

It’s challenging to develop good Key Performance Indicators (KPIs). It’s easy enough to come up with the right metrics, but clarifying robust objectives is more difficult. It’s also easier for e-commerce sites to develop KPIs with strong objectives.

But most sites don’t sell stuff, so why do they exist? They exist for good reasons and those good reasons need to be expressed in clear and definitive terms. That’s the hard part. So, make the objectives clear and measurable, describe what “good looks like,” and select appropriate metrics that measure outputs and outcomes. So once that is done, is that they end of the story? Not really, it’s just the beginning.

Once overall site goals are in place and appropriate KPIs have been defined, the next question is: which sections of the site are working well and which ones aren’t? If we don’t know the answer to this, then how can we focus our efforts properly on site optimisation? This is why we need not just site goals but page goals as well. A page goal defines what that page is trying to achieve. It answers the question, “Why does this page exist?”

Stating clearly why a page exists and its objectives are is a very useful exercise, particularly when you are designing a page. Page goals help you focus what the page is trying to achieve and feed directly into the development of the information architecture of the page – for example, at wireframe stage. Page goals are particularly useful in circumstances when there are multiple stakeholders involved all trying to get a piece of the action on a page; without page goals you can end up with pages that don’t work particularly well for the business or website visitor.

In the same way as for defining overall site goals, page goals should be as clear and precise as possible. Something like “To help users achieve their goals” doesn’t really cut it. Which users? Which goals? In what way? It has to be clearly defined. Once it’s been clearly defined then a measurement framework can be developed for each page that describes how the page’s success can be assessed. Once the measurement framework has been developed, then measurement systems can be configured to include the right kind of metrics in the right kind of way. With page level measurement frameworks,  it also important to consider the context of the page itself. How do people get to the page? How far is it into a customer journey?

Consider the classic product page in an e-commerce site. A product page has a tough job. The primary purpose of a product page is generally to persuade a website visitor to add a product to the basket. To do this it needs to provide all the information required in a clear and concise way. To measure the success of the product, you could look at metrics such as the add-to-basket rate. You could also measure the effectiveness of different components or tools on the page in terms of how they influence the add-to-basket rate. But often a product page on an e-commerce site is also the landing page. Often it’s the first page that a visitor sees on a site. For them it’s also the home page. So, the product page also has the goal of building trust and consideration for new visitors. It doesn’t just have to persuade the potential customer to buy the product; it has to persuade the customer to buy it from you. By identifying these additional page goals, you also identify the need to measure success using additional metrics, such as the bounce rate for new visitors.

Not every single page on a site necessarily has to have page goals, but certainly they should be in place for each different type of page or section on a site. Page goals are useful to help define what good looks like at the micro level and to ensure that your measurement frameworks are measuring the right things in the right way.

This article was originally published on ClickZ

One of the big challenges of providing usability, accessibility and analytics consultancy services to help businesses improve their customer experience, is to ensure that information is digestible and therefore more readily usable and practical.

There is a trend, particularly in the web analytical arena, for some consultancies to simply create more and more metrics for businesses, rather than working on their clarity. Unfortunately, there is also a tendency for many people to suffer from ‘data blindness’ when confronted with rafts of metrics churned out by multiple tools and technologies. Advertising campaign management, web analytics, CRM, and other intelligence tools should enable businesses to interact better with their customers, but for many actually understanding all that information – particularly across wide portfolio websites – proves very frustrating. Read more…

 This article, written by Neil Mason, was originally published on Clickz.com and is republished here with permission.

I’m increasingly convinced that the issues that most businesses face around the successful deployment of analytics in their business are not to do with their technologies but to do with their businesses processes. That view was reinforced this week when I was running a workshop with a group of students studying on a Masters Programme in Internet Retailing. Read more…

 This article, written by Neil Mason, was originally published on Clickz.com and is republished here with permission.

Jim Sterne, the author and Chairman of Emetrics, was in London last week and I had the opportunity to catch up with him at our offices. During our conversation we talked about the way that the market is developing and what the differences were (if any) between what is happening in the digital marketing optimisation space on my side of the Atlantic compared to his. As we were talking I kept thinking back to a simple model that I developed a number of years ago to describe where organisations are in the development of their digital marketing measurement and optimisation capabilities. These days it would be fashionable to call it a “maturity model”.

My simple maturity model has three main stages:

• Performance tracking
• Process optimisation
• Customer centricity

Performance tracking

In the first stage of their development organisations are focussed on performance tracking. The challenge here is to “get the right numbers right”. This is laying the foundations for further capabilities to be developed and the main activities revolve around deifying the needs of the business, setting success criteria such as Key Performance Indicators (KPI’s) and getting the right measurement technologies in place and getting them working properly. This will involve a lot of effort with specification and configuration issues.

At this stage in the model, organisations might be wondering whether it’s all worth it. The “effort to insight” ratio is high. The amount of value being gained from the data may seem to be low compared to the amount of heavy lifting required to get it. However, it’s vitally important that this bit is got right otherwise it’s hard to move from this stage of the maturity model to the next.

Process optimisation

Once the right numbers are right, organisations can start to use the data to make better decisions. The application of insight into the business creates an opportunity to optimise processes and to begin to create some return on investment in the putting the measurement capabilities in place. So the focus shifts from tracking to optimisation of the core digital marking processes such as acquisition and conversion.

The process of optimisation is driven by the “test, learn and adjust” approach and requires more than just good data and some smart technology. It also needs the right organisational culture backed up by sounds business processes. These processes needed to be embedded into the organisation during the Performance Tracking phase and they guarantee data integrity. Nobody likes making decisions on dodgy data and you can’t optimise marketing processes without making decisions about what’s working and what’s not.

Customer centricity

During the Process Optimisation stage, organisations still tend to be quite “site centric”. They are focussed on how to optimise a series of processes, tweaking conversion rates, improving satisfaction scores and so on. The next stage in the model is moving from being “site centric” to being “customer centric”.

The main difference I think is the difference between asking what the conversion ratio was last week and asking how many new customers there were last week and what’s their expected value in the future. Customer centricity is about looking at the digital channel in the context of an organisation’s overall relationship with the customer rather than the other way around.

The main challenge here is to be able to get a multi-channel perspective on the customer. To do that you need multi-channel data, bringing together where possible data from offline systems and online systems.

The conclusion I came to from my conversation with Jim was that the difference between what is happening over here and what is happening in the US is essentially down to the distribution of where organisations are in the maturity model. There are some companies in the UK and the rest of Europe doing some really interesting stuff, it’s just that there are more of them in the US. But that’s OK, sometimes it’s best to go second, you can learn quicker that way!

Video interview with Jim Sterne

This post originally appeared in Applied Insights’ events section. Foviance acquired Applied Insights in November 2008, with Neil Mason joining us as Director of Analytical Consulting. As part of this acquisition, we’ve incorporated Applied Insights’ events list into our own.

This year Neil was invited by Jim Sterne to be the conference chairman and a keynote speaker the Emetrics Marketing Optimisation Summit in London.

As well as fronting up the proceedings over the two days and trying to keep the conference (and its speakers) on track, Neil delivered a keynote presentation looking at the development of web analytics and marketing optimisation practices within organisations called: “To Marketing Optimisation and Beyond!”

This article, written by Neil Mason, was originally published on Clickz.com and is republished here with permission.

Last time I shared some thoughts about the process of selecting analytical systems, particularly web analytics systems. These included:

• Being clear about your goals
• Defining the KPIs and key metrics
• Defining the business processes
• Writing a challenging business requirements document

But selection is only half the battle to successful adoption, there is also the issue of actually implementing and getting business value from it. In the past I have seen systems implemented that work perfectly well but are not felt to deliver much in the way of value, simply because they are being underutilised (for whatever reason) relative to their potential. So, what are some tips for the successful implementation of analytic systems?

Write a plan

This may seem like a case of stating the obvious but the point that I would make is to write a type of business plan rather than a technical systems implementation plan. The plan needs to consider all the environmental and organisational issues as well as the technical issues.

On one level there are all the technical issues that need to be thought through such as how is the site going to be tagged or how are the log files going to be harvested? The devil is in the detail with these sorts of issues and careful consideration needs to be given to them upfront to avoid pain later.

One another level the success of the system implementation will also be judged on whether people are using it and whether they are getting any value from it. This is where the organisational and environmental factors come into play. Different types of organisations and functions are likely to have different attitudes to the adoption and use of analytical reporting systems. So what’s the plan to get them engaged on the level they need to be engaged at?

Find advocates

As part of the plan you may want to think about who your advocates are likely to be. These are the people who “get it” and will be willing adopters of the new system and the opportunities that it will offer them.

One of the mistakes I made in the past when trying to roll out a new analytical system was to try and roll it out on too wide a front. Too many people were involved and at the time it was like trying to push water uphill. Since then I have found that a useful approach is to find group of individuals or a function that are happy to be early adopters and work closely with them in the early phases to get the system actually being used in the business.

In the case of a web analytics’ system you may work with the advocates to get some of the more sophisticated features of the system up and running such as campaign tracking capabilities. Working with these advocates can then help you do develop useful case studies that you can use with the rest of the business in seminars, training courses and the like.

Deliver appropriate training

In order to get the most out of a new system everyone needs some training even if they are not going to be actually using the system itself. Training can often focus on how to “drive” the system itself, a bit like learning how to drive a car, but it’s also useful to coach people in what to do as a result as well, and that’s where the case studies you’ve built up with your early adopting advocates come in handy.

In many cases systems can “push” reports out to end users via email or on an intranet, which is great. But do they end users always know what these reports are telling them and what they should be looking for? I’ve argued in the past that an analyst adds value by adding interpretation but if this isn’t possible all the time at least think about how to coach end users in how to interpret the data that’s being delivered to them.

Demonstrate value and celebrate success

The proof of the pudding is in the eating and as I mentioned last time you only really know what you’re going to get with a new analytics system once you’ve got it in and up and running. It’s important that the value of the investment is demonstrated to the business. This may take different forms depending on the business but there should be some early winds that you can point to in order to demonstrate the success of the implementation. These may be new approaches to campaigning that are being implemented or improvements in conversion in certain areas. Whatever they are, add them to your library of case studies and use them to show the business value every opportunity you get!

Next week I’m heading over to Search Engine Strategies in New York to take part in the ClickZ track. I’m really looking forward to catching up with some old colleagues and hopefully meeting new ones. I’m also looking forward to taking a look at the search analytics scene and will reporting back on what I find next time round. Till then…

This article, written by Neil Mason, was originally published on Clickz.com and is republished here with permission.

There was a bit of research publicized recently about organizations’ satisfaction with web analytics systems. It reported that 62% of the respondents from these large companies were happy enough with their web analytics package that they would recommend it to others. 62% seems like a pretty good number to me but what about the other 38%? Are they not getting what they need from their package and if not, why not?

I’ve worked with a number of companies over the years helping them to select and implement analytical software including web analytics packages. Indeed I went through the painful process myself, implementing a tag based ASP solution across a dozen sites across Europe and integrating it in with other information and operational systems. As a result of the work I do with clients, I have heard lots of stories about why they want to move from their existing package to another one.

Some of the common themes that I come across are:
The system doesn’t deliver any value to the business…
The system sits in isolation and it’s difficult to get it to integrate with other data…
The system doesn’t appear to work properly and deliver what’s needed…
Nobody really knows how to use it…
These themes are usually the symptoms of a poorly thought through selection and implementation process for the existing system and may not always be a problem with the system itself. Often these issues stem from a lack of clarity about what is really needed in the business and also an underestimation of the associated effort required.

There can be a tendency in the process of selecting analytical systems to be led by the technology and the feature set. As I mentioned in the last article, funky overlays and pretty dashboards might make the data look good, but you also need to ensure that you can get at the right sort of data you need in the right sort of way.

Here are some thoughts about how to minimize the risk of being one of the “38%”.

• Be clear about your goals
• Be very clear about what you are trying to do online and the reason for the investment in the web channel. From this all else flows.
• Define your KPIs and main tracking metrics

Once the goals are visible, it’s possible to identify what the channel Key Performance Indicators (KPIs) are. In addition, there will also be a number of other important tracking metrics as well, which are not as strategically important as KPIs. It’s important to recognize that KPIs are not always metrics that come out of a web analytics package. In fact, in my experience they rarely are.

Define the business processes

As well as defining the important metrics, it’s important to map out and define the important businesses processes. How are campaigns managed? How is the site development process managed? The tools should help the business processes and you shouldn’t need to change your business processes to fit the tool, unless it is clearly a much better way of doing it.

Write the business requirements document

This is the document that goes out to potential vendors. The point of the document is to clearly set out your needs and to invite the potential supplier to articulate how their system can meet them. In my view this document should enable the best potential suppliers to shine through, so don’t be too prescriptive in approach. The kind of huge document that requires huge amounts of detail input on every single aspect of the system does not help at this stage. You hate writing them, the vendors hate responding to them and then you hate reading them. And even then, there is no guarantee that you will be able to see the wood from the trees anyway.

Keep the requirements document concise, clear and open. Invite vendors to make an effort to differentiate themselves. A vendor once told me that the best brief they had had from a prospect was “This was what we are trying to do, tell me how you can help us do it better” and that was it.

Get it down to two or three

From the various responses that you get, you need to whittle it down to two or three potential vendors to come in and pitch for the business. Don’t invite them all. Look for quality not quantity in the response documents that you get back from them. First of all, have they answered the brief? Have they thought about what you asked them to think about? Or have they merely cranked the document out of the proposal machine? Do they demonstrate an understanding of the business and how they can help? Does it look like they want your business more than the next person?

Test it out if you can

In an ideal world you should test at least your preferred system, or preferably two systems side by side to see them in action. Analysis and reporting systems are “experience products”, you only really know what you are going to get when you’ve already got it. Having real hands on experience will be invaluable in helping you decide whether this is the one from you and you are going to be one of the “62%” of organizations.

Selection though is only half the battle. The other half is getting the system in, working properly and being used to its potential. Next time, I’ll being sharing some thoughts on successful implementations. Till then…