“All generalisations are false, including this one.”

Here he was talking about segmentation. Generalised metrics such as overall conversion ratios, average time on site, and so on are largely useless. They only make sense when the data is segmented and the underlying patterns and differences can be understood. The same is for survey data as well. Metrics such as NPS can vary wildly among different customer segments and so these segments need to be identified and looked at separately.

“Few things are harder to put up with than the annoyance of a good example.”

A great insight into the power of evidence-based decision making. Solid facts, presented well, are hard to ignore. Twain was probably also thinking about the need to have an approach to testing and experimentation when he said this. There’s no better way to silence the HIPPO (highest paid person’s opinion) than to have concrete evidence from testing about why one approach is better than the other.

“It’s no wonder that truth is stranger than fiction. Fiction has to make sense.”

Sometimes the data doesn’t make sense and the skilled analyst needs to understand why. Is the data telling her something new? Is this real insight or is there a problem with the data? Often, if it doesn’t seem right, then it probably isn’t right, but occasionally there may be something that the business has missed, the competition has missed, and there is real opportunity to be exploited. Just make sure you’ve checked the data thoroughly before you tell the CEO!

Twain also had some great advice for budding analysts:

“The more you explain it, the more I don’t understand it.”

This is a great piece of advice. Basically, keep it simple, but don’t be simplistic. A good analyst will be able to present quite complex notions in ways that are easy for people to understand. This is the art of storytelling. Developing compelling narratives that engage the audience is quite a skill. There’s an important role for data visualisation here as well. It’s harder to create simple charts and graphics than to scatter lines and bars all over a PowerPoint slide. That’s why an analyst also has to be an artist.

“I was gratified to be able to answer promptly, and I did. I said I didn’t know.”

I think it’s OK to say that you don’t know; it’s better than bluffing…

“It is better to keep your mouth closed and let people think you are a fool than to open it and remove all doubt.”

…and being found out later, as long as you say that you’ll find out the answer and come back to them. And then make sure you do.

And finally:

“The most interesting information comes from children, for they tell all they know and then stop.”

Don’t be afraid of silence, particularly if you’re presenting to your audience. Deliver your insight and wait for a response. Maybe ask them a question, and then wait for a response. “Is that something you recognise?” and “Does that fit in with what you know?” are all ways of seeing whether what you’re saying is hitting the mark or missing the mark. It’s better to find out in the course of a conversation that what you’re saying is not falling on fertile ground rather than just to be ignored later. Maybe you need to get more evidence or maybe you need to explain the point in a different way.

So that’s why Mark Twain was an analytics guru!

This article was originally published on Clickz

Last time I looked at some of the characteristics of data collected from surveys, particularly data collected from surveys run on websites where you have no control on who is answering the survey. Generally this lack of control can cause some bias in the data which can cause some issues if you are looking at the aggregated reports. For example the data on the profile of visitors (i.e. gender, age etc) that you collected from survey data may not actually reflect the true profile of visitors to your site because of the different propensities of different groups to respond to surveys. So, does that mean that survey data is useless? Not really but it does means that it needs to be handled with a bit of caution.

One way to reduce the impact potential biases in the data is to trend the results over time. I’m always think that survey data is most useful when you have it running continuously anyway as it means that you have a constant monitor of the health of the site and you can refer to it to assess the effect of all sorts of marketing and product development activity. Having a continuous dataset also helps to reduce some of the bias. Say for example, that your survey shows that the age profile of visitors to your website is 40% under 35 and 60% over 35. We know that generally younger people are less responsive to surveys than older people and so we might suspect that there is a bias in the data towards older people. If however, 6 months later you look at the data and it shows that the profile has changed and that it is now 60% under 35 and 40% over 35 then, all other things being equal, whilst we still can’t be sure that the profile is absolutely correct, we can be reasonably confident that there has been a change in the profile over time and that the profile has got older. If we wanted to we could also check whether the change had been statistically significant or not.

Another way of reducing bias in your data is to segment your data. In fact I would say that you absolutely have to segment your data to make it useful and to understand it properly. So whilst I might not be confident that the profile data is properly representative of the reality, I can still use the profile data to look for differences in some of my key metrics such as customer satisfaction or the Net Promoter Score (NPS). I can compare satisfactions scores amongst the younger age groups and the older age groups to see if there are any significant differences and because there often are, I should always be looking at these key metrics amongst key segments of the site’s visitors. This is because changes in the visitor profile of the site can have a significant impact on the changes in these key metrics. Let me give you an example.

As I mentioned last time you can see differences in metrics like satisfaction score or NPS amongst different segments depending on their familiarity with the site or the brand. Often people who are visiting your website for the first time will have lower scores for satisfaction and NPS than those who have visited before. Let’s assume that you have been running some campaigns either online or offline and have driven a significant amount of new traffic to the site. The survey you’re running on the site will probably reflect the increase in new visitors and as a result it’s possible that the overall satisfaction score will go down. This not because people are overall less satisfied with the site experience but because you have a greater proportion of people answering the survey (i.e. first time visitors) who generally tend to give lowers scores. Nothing may have actually changed in the site experience itself, the only change has been in the mix of visitors to the site. In fact, the satisfaction amongst first time visitors can have stayed the same and the satisfaction amongst repeat visitors also can also have stayed the same but apparently overall satisfaction can appear to have gone down.

So, on the face of it online survey based data looks to have some serious issues with it. However, by understanding the source of these issues and interpreting the data wisely can ensure that you can get some real value from this rich source of customer insight. And remember…segment, segment, segment!

In the previous two columns I have been looking at different types of segmentation strategies, mainly dealing with what segmentation is, the different types of segmentation strategies and the role each type can play in building up a core understanding of your customers or prospective customers. So once you’ve decided what to create the segments on, the question then becomes about how to create the segments. Remember with segmentation what we are trying to do is to create groups of people who have something in common.

When it comes to creating segmentations, there are two main alternative approaches:

• Deterministic segmentation strategies
• Discovery segmentation strategies

With deterministic segmentations, the user or customer segments are based on some kind of hypothesis and then the data is analysed to see whether the segments are interesting and useful. For example, demographic segmentations often tend to be “deterministic”. You may segment your customers on the basis of gender and age in the belief the criteria are useful and interesting. Also most segmentation on web data that’s done at the moment is done this way. Most of the web analytics tools that people are using have some kind of segmentation capabilities built into them, allowing you to start to create hypotheses about what might be useful segments to analyse, understand and track. For example you might be interested in looking at the differences in behaviours based on the number of times people visited the site, or the channel they came in on, or the search terms used. Deterministic approaches can be successful but they can also involve a lot of time in analysis, particularly when dealing with large and complex data sets. Many iterations might be required in order to indentify segments that are meaningful, interesting and useful. This is where the power and functionality of your analytics tools becomes vitally important. If it takes you ages to create a segment and to see the results, then this will inevitably mean that you won’t arrive at an optimal solution.

Discovery based segmentation approaches use statistical and data mining algorithms to look for differences in user behaviour. Typical methodologies used here in segmentation studies include cluster analysis, neural networks and decision trees. Methods such as cluster analysis look for statistically meaningful differences between different users groups based on the data that fed into the analysis process. This is a massive area as there are many different types of segmentation techniques. Even when talking about cluster analysis, there are many different variants of cluster analysis such as k-means, hierarchical, two-step and so on. Each approach has its strengths and weaknesses and even within a single variant of cluster analysis there are many different ways that the analysis can be run. Although these solutions can be viewed as very technical, they is as much analytical “art” behind a successful outcome as there is “science”. Once those groups have been determined further analysis is done to profile the groups to understand what those differences are and whether they are meaningful or not. Just because something is statistically significant, it doesn’t mean that it is necessarily commercially significant!

Discovery based methods can yield user segments that may not be immediately obvious from the data. This is one of the benefits of using this type of approach. Quite often in the work that we have done using these types of techniques on web data we find that some of the more interesting and valuable segments are quite small, and this is because web analytics data typically contains a lot of noise from people who only ever visit the site once or twice and do nothing of any value. However, discovery based approaches require specialist skills and are highly iterative and consequently are more likely to be more costly in terms of both time and money.

In my last column, I took a look at the meaning of segmentation and the different types of segmentation strategies available to digital marketers. There are three main types of segmentation; demographic segmentation, behavioural segmentation and attitudinal segmentation. But which one is best? The answer is that it really does depend on what problem you’re trying to solve.

Demographic segmentation strategies have traditionally been used by marketers for targeting purposes. Customers or prospective customers are classified according to different demographic criteria and are then selected for different types of marketing activities or communications. Often predictive models can be used to predict which segments are most likely to respond to which types of campaigns based on their previous history. The ability to identify potentially lucrative segments and then target them can be very powerful and result in much higher returns on marketing investment.

Demographic segmentation could be a useful approach for digital marketers but it does rely on the type of data on customers and prospects that can be collected. Strategically it can be important to understand which type of people are likely to be interested in your product or service or to shape your product or service to appeal to a particular demographic segment or segments. The data for developing the segmentation might come from existing customer databases or if you don’t have a customer database then it might need to be collected using other data sources such as online surveys. Integrating survey data with web analytics data could help you to understand for example conversion rates amongst different demographic groups. Media planning tools can then be used to refine the acquisition strategy orientated around those demographic groups with the highest potential.

Using demographic segmentation approaches can be as useful online as they are offline but collecting the data can be a problem. However, we are generally not short of data on how people behave online and so behavioural segmentation approaches can be not only powerful but easier to adopt. Behavioural segmentation lies at the heart of most personalisation and behavioural targeting techniques whether they are based on relatively simple “rules based” approaches or more complex models and algorithms. The data for behavioural segmentation is readily available in your web analytics system and these days most web analytics tools give you the ability to cut the data a number of different ways. So there really isn’t much excuse nowadays not to start to look at segmenting your audience or customers based on how they behave on your website or how they interact with you over a period of time.

Some simple behavioural segmentation strategies can be very powerful. Optimising landing pages based on source of acquisition is a simple but effective behavioural segmentation approach. Creating different experiences base on the number of times that someone has visited the website is another. One of the classic behavioural segmentation strategies is Recency Frequency Monetary (RFM) analysis. Developed originally by catalogue retailers, in RFM customers are categorised according to how recently they transacted with you, how frequently they have done that in the past and the monetary value of those transactions. The high recency, high frequency, high monetary value group are your most valuable customers, (for example an airline’s Gold Card customers) and the way that you would market to them would be different to other groups. On the other hand a new customer (high on the recency scale but low on the frequency scale) presents a different opportunity and the key thing is to get them to buy or transact again.

The challenge of applying online behavioural segmentation approaches is to manage data across different systems either doing that manually or by having more integrated solutions. Once gain this is becoming easier for digital marketers as many of the web analytics providers have interfaces to other marketing systems (such as email tools) to enable these types of behavioural segmentation strategies to be implemented.

One of the limitations of behavioural segmentation is that whilst you might know what works there may not be a lot of insight into why it works and consequently how it might be improved. Attitudinal segmentation involves getting in the mindset of your customers and understanding what makes them tick. This allows you to potentially develop different strategies for different people based upon their attitudes and opinions about your product or service rather than how they interact with you. This type of segmentation lends itself to applications such as design work where you are trying to develop solutions that are appropriate for different groups of people based on their needs, goals and ambitions.
Whereas we have data on behaviours in abundance in our digital marketing world, we rarely have abundant data on our customers or visitors. As with demographic data we need to go out and collect that data from sources such as surveys or to get really deep insight other techniques such as depth interviews or focus groups. As a result the data that feeds into attitudinal segmentations may be rather sparser than that for behavioural segmentation approaches but can be richer.

So, which is best? As with most analytical techniques, it depends on what problem you are trying to solve. For developing acquisition strategies, demographic segmentation techniques can be really useful. For improving design and conversion, attitudinal segmentation feeding into persona development can play a role and for improving retention and customer lifetime value, classic behavioural techniques such as RFM can be powerful. Whichever approach you use though, there really isn’t any excuse these days for carrying on with “one size fits all” digital marketing strategies.

One of the things I like about my job working is a customer experience consultancy is that I’m surrounded by people with a very different outlook on life. Our user experience consultants tend to come from a behavioural psychology background and are great at using qualitative research techniques such as lab testing, eye tracking and ethnographic studies to get into the mind of users and to understand what makes for a good or bad experience. That’s obviously a different set of skill and tools from our quantitative, analytical approach to solving problems using vast quantities of data. Each approach is complimentary to the other with quantitative data good at asking the “what” and “when” type questions and qualitative techniques good at helping to understand the “why”.

Every now and then we get into one of those interesting conversations about which approach is best for solving a particular type of problem. Last week ones these conversations was around the topic of segmentation and which types of segmentation are best for addressing particular issues. Segmentation is one of those popular words that’s used a lot these days in the digital marking world and is usually means different things to different people.

Segmentation is the process of creating groups of individuals (customers, website visitors, prospects etc) that have something in common. Importantly what one group has in common is then different to the other groups. The purpose of segmentation is to make you, your marketing communications, your website experience, your product offering on so on more relevant where possible to these different groups. But how are these groups defined? There are three main ways:

• Demographic segmentation
• Behavioural segmentation
• Attitudinal segmentation

Segments can be defined by demographics, ie based on who someone is. Typically classical marketing approaches to segmentation use demographics as the basis as it can then be used for targeting purposes. Demographic segmentation in online can also be useful. For example, “gender” can be a useful segmentation split as the way that people behave online can be very different depending on whether they are male or female. So be able to segment your audience by gender, age, income etc can be really useful.

Another approach to segmentation is behavioural segmentation. This is not classifying people according to who they are but on the basis of what they do. This type of segmentation approach is very popular in digital marketing as it’s quite easy for us to understand how people behave as we have loads of behavioural data. Again it can be a very powerful technique to group people according to different behavioural criteria and to use that knowledge to improve the effectiveness of campaigns or to present different website experiences. For example, the way that people who are on their first visit to a website is often very different to the way that they behave on a subsequent visit and their needs are also often different. So why not present them with a different experience? Behavioural segmentation lies at the heart of personalisation.

Finally attitudinal segmentation is about classifying people not according to who they are, or what they do, but about what they think. Attitudinal segmentation is about getting into the minds of customers and understanding what makes them tick. People of different genders and ages may have similar needs when it comes to interacting with product and services, they may be trying to pursue the same goal or trying to achieve the same outcome. Often attitudinal segmentation is used for the development of “personas” which are used as tool to help designers get closer to the people they are designing for.

So which type of segmentation is best? Well, of course, the answer is that “it depends”. What problem are you trying to solve? What will you do with the segmentation when you’ve got one? The other questions then are “What data do I need and where do I get the data from?” I’ll be looking at the answers to these questions next time. Til then…

This post originally appeared in Applied Insights’ events section. Foviance acquired Applied Insights in November 2008, with Neil Mason joining us as Director of Analytical Consulting. As part of this acquisition, we’ve incorporated Applied Insights’ events list into our own.

Applied Insights ran a one day workshop in Predictive Analytics in association with the Emetrics Marketing Optimisation summit on 22^nd May at the Hotel Russell in London. A course outline is below.

Please contact us if you would be interested in joining one of our courses or developing a customised in-house training session on predictive analytics.

Predictive Analytics – course outline

An Introduction to Data Mining and Predictive Analytics is a one day workshop covering the foundations of this innovation marketing analytics discipline. During the course of the day you will gain a thorough familiarisation with some of the key principles and methodologies of data mining and predictive analytics and learn how to apply them to common marketing problems such as:

• How can I predict campaign response?
• How do I segment my website visitors or customers?
• How can I anticipate possible customer defections?

In this one day interactive course we will cover the following topics:

Introduction:

• What is data mining and how is that different to predictive analytics?
• How organisations are currently using data mining and predictive analytics across their businesses and to solve particular marketing problems

Processes and implementation

• How to go about a data mining/predictive analytics project
• An overview of a standard industry process (CRISP-DM)

Methods and applications

• An overview of the main types of data mining and predictive analytics applications:
  • Forecasting
  • Segmentation
  • Classification
• An introduction to main methodologies such as:
  • Time-series forecasting
  • Regression analysis
  • Decision trees (CHAID, CART and so on)
  • Cluster analysis
  • Neural networks
• Case studies and examples of how these techniques are used and deployed in both online and offline marketing is areas such as:
  • Retention modelling
  • Conversion propensity modelling
  • Visitor segmentation

This post originally appeared in Applied Insights’ events section. Foviance acquired Applied Insights in November 2008, with Neil Mason joining us as Director of Analytical Consulting. As part of this acquisition, we’ve incorporated Applied Insights’ events list into our own.

At this year’s Emetrics Summit in San Fransisco, Neil will be presenting a session in the “Advanced Analytics Track” entitled ‘Cutting through the NOISE: Applications of data mining and predictive analytics’.

The presentation will be looking at the application of techniques such as segmentation and propensity modelling to better understand website visitor behaviour.

A few weeks ago I took stock of the web analytics market, particularly looking at some of the key trends in Europe. This week to get a sense check from the position of someone in the US, I turn to a fellow WAA Board Director and friend Avinash Kaushik. Avinash is also an Author and one of the leading thinkers about web analytics and where it’s heading, having actually “been there and done it” previously at Intuit software.

This is what Avinash had to say.

Avinash, you had a busy year in 2007. What were some of the highlights for you?

It has indeed been a hectic year, becoming an Independent Consultant and Analytics Evangelist role at Google and publishing Web Analytics: An Hour A Day in June. Along they way speaking at conferences and running around the country became normal! Oh then there is the blog, Occam’s Razor, my baby (!), that took more time than I could ever have imagined.

I think the biggest professional highlight has to be the book. In five months sales have vastly exceeded my expectations. Since all of my proceeds go to charity (The Smile Train, Doctors Without Borders) it has meant a nice amount raised for them.

The book is a great primer and reference document for all things “web analytics”. But in this fast moving industry, isn’t it a risk writing a book? Are there some parts of the book that you think you might have to rewrite soon?

The core of the book I think will stand the test of time (and by that I mean five years at most! ). But there are many sections I would update. The book has been out only five months but I would add new things to the SEO section. Ditto for blogging metrics, I have slightly changed two of the six in the book and added a brand new one. I touch on Social Media but when I write the next version of the book I think things will be more settled and I can add more interesting things.

New tools will come with time, as will new sources of data and my book, or and those of others, will accommodate for that. But the biggest goal of Web Analytics: An Hour A Day is to teach you a new way of thinking, that I think will be relevant for quite some time to come.

All that said Willem from Wiley was over the other day asking me to start work on the next version!!

What do you consider to be some of the key industry developments to have been in 2007?

I get the distinct feeling that we will look back at 2007 and remember it as a turning point, a good one, for the industry.

Why is that?

Every site in the world seems to have Google Analytics – a leading indicator that even the most common person with tangential interest in data now has access to a great web analytics application. More interest translates into more mind share.

The industry has consolidated quite a bit. Omniture has built on top of its already impressive growth by acquiring Visual Science (/WebSideStory / HBX), in addition to Instadia (Analytics + Surveys), TouchClarity (Behaviour Targeting) and Offermatica (Multivariate Testing). This year all roads seemed to lead to Salt Lake City!

WebTrends is going through some temporary management turmoil, but with its excellent set of solutions I expect them to come back strong.

There were more web analytics consultancies launched, more than on you can count. Ditto for web analytics conferences. Actually a real interesting trend was how many non-analytics conferences had “web analytics day” or “web analytics pre-intensives” – a real sign of growing demands.

It was also a year of Web Analytics 2.0. An expansion of the core definition of what web analytics is, stretching is beyond just clickstream to include qualitative research, testing, competitive intelligence, multiple dimensions of outcomes etc.

So what are some of the key drivers?

Many, if not all, of the trends above were driven by a singular phenomenon: The web is becoming serious business.

It seems odd to say this in 2008 but in many companies web, and web analytics, have been a silo that someone else is taking care of. Websites are becoming the most important customer touch point and the most important revenue generator even for businesses that are not first of mind.

The consolidation in the industry, the increase in interest (tools or conferences) and expansion of the definition is a reaction to the demands now being placed driven by a desire to move beyond printing reports (to perhaps printing money!).

How would you assess where the web analytics industry is at the moment from the point of view of software vendors?

Full of opportunity.

Money and fame awaits all. Well at least those who are willing to work hard.

The vendors have done well thus far, mostly, but they are still scratching the surface of what is possible. Many big websites still don’t use web analytics. There are many growth opportunities in the Software market (aside from the current favourite child: hosted). We are not even scratching the surface of integration with data from other parts of the company and other tools should we decide that web analytics is not a silo but a part of “Business Analytics”? So there is a lot to do and appropriate financial rewards for companies that help accelerate the move beyond clickstream.

What about the people side, i.e. the end users and consultants?

There is a read dearth of skilled practitioners in our industry. And that has stunted the amount of progress that can be made (because the 10/90 rule still applies – spend $10 on software/services and $90 on people who can actually analyze data and produce insights). If you are a skilled person, you can name your own salary (but make sure you are on the web analytics 2.0 continuum and not 1.0), and if you are someone who wants a great Analytics career then now you know where to find it.

Consultants will thrive in any field where the rule is 10/90, because they can bring their expertise to bear on the $90 part of the equation. Additionally because of increase in the demand you are noticing many more consultancies (mom and pop and grandpa included), and an interest from the “big boys” for mature web analytics consultancies (example: our good friends Zaaz acquired by WPP). To make optimal amounts of money Consultancies, like other companies, are finding that they can’t be a one trick “let me parse your log files” pony. They are being forced to evolve into areas such as multivariate testing, competitive intelligence, usability etc.

What are some of the key trends that you see at moment? Where’s the market going?

The problem with Web Analytics 1.0 is that it is an exercise in data torture and reporting with long lags in taking action (if any). Data torture needs to get automated and expanded, decision making needs to get automated; people need to be left for smart hard things (vs. what happens today!). Smart companies will start to exploit more things like Multivariate Testing, Onsite Behaviour Targeting etc because in each case you are leaving humans to understand customers and create content and you are letting intelligent solutions create the right customer experience based on data. Won’t happen overnight, but are on this train for good.

I also believe that 2008 will see a more serious attempt to get Web Analytics to become a part of “Business Analytics”. We are still a silo in most companies (data and people!). We will see more collaboration and innovation in helping web data become a core part of the company data to truly give end to end visibility (and maybe the holy grail of multi channel analytics / impact). Won’t happen all in 2008, but we might get serious.

I am optimistic that we don’t have untouchable islands of data like we do today. Search Engine Optimization, RSS, Social Media, etc. They are all becoming mainstream yet the current generation of tools mostly stink at tracking them. You can track them, but if you are willing to row your leaky boat all by yourself to that island. I think this will change.

Oh and we are not done with consolidation in the industry.

It’s going to be fun!

I reckon so, thanks Avinash

It’s a new year and with a new year come new resolutions. Other than the usual New Year’s resolutions to eat less and to exercise more, I have been thinking about the consulting work we do with our clients and what resolutions we should be making in that area. I have come up with two that I thought I would share.

Think strategically, Act tactically

This notion has been playing on my mind for a while. A catalyst was listening to Jim Sterne speak at the Emetrics conference in Washington DC in October. Jim was talking about the need to solve people’s most immediate and pressing problems before you can then go on and show them all the amazing opportunities they might have in their online business. In our own work one of the things that we are conscious of is an organisation’s ability to execute from insight. So, if an organisation knows something, can they do anything about it?

We have seen occasions in the past where we have been engaged to work with organisations on developing strategic pieces of insight which the client then struggles to leverage due to functional or organisational constraints. I believe that it’s still necessary for analysts and consultants to “think the big thoughts” but they need to be translated into little actions that are digestible for the business in question. As consultants and analysts this is what we all try to do, my resolution for this year is to make this a core feature of the way to do business.

Segment, Segment, Segment

Segmentation has perhaps been one of the buzz words for 2007 and rightly so. In mature online economies, the ability to segment and target market to sub-groups is increasingly an important way to continue to grow and develop the business. However, not enough businesses are segmenting their online visitors and customers, and are still operating a “one size fits all” policy on their website. Larger and more complex websites will inevitably have a wide array of different visitor segments who are coming to the website for different reasons and who want to do different things.

More tools are available for understanding visitor segments from a behavioural, demographic and attitudinal perspective. In addition marketing technologies have developed to allow organisations to target different visitors with different messages and content using rules-based and automated profiling techniques. So, my second resolution is to continue to encourage businesses to invest and adopt these tools to allow them provide increased relevancy and improved customer experiences. I believe that those that do will stay up with the game, leaving behind those that don’t.

So those are a couple of my resolutions. What are yours?

Best wishes for a successful 2008.

In my last article I outlined my belief that what we call ‘web analytics’ is becoming a more diverse and complex field. What we have traditionally considered to be web analytics has been the analysis of site behavioural data captured, processed and reported on by a proprietary system designed to do just that. But as the online channel evolves and becomes more complex , the tools to help us understand what’s happening must also evolve and become more complex. In some areas, such as in the case social media, this may mean the development of new tools. In other areas it may mean the application of old tools to this new channel.
One of the areas that we work in a great deal is in the use of data mining and predictive analytical techniques. I first got started in this area about 15 years ago when at ACNielsen using these types of methodologies to help clients to try and figure out which half of their advertising money they were wasting. I have a book on my bookshelf that was published 25 years ago on the use of model building techniques in marketing. So the techniques aren’t new but what is relatively new is the systematic use of these techniques in the online marketing space.

I think that there are some reasons for this. Historically our main concern has been on managing the vast volumes of data and wrestling out of the web analytics systems a few numbers that told us how well we were doing and that we could do something about. Also, in the past, the natural organic growth in the channel has meant that we have not been faced with the need to scramble for market share and to fully optimise our business processes. And to some extent, we have not been asking the right questions. This is now changing. We understand our few numbers and we want to know more. The online world is far more competitive and we are beginning to ask questions that go beyond the limits of our traditional analytical tool set. Questions like:

• How do I understand the effects different marketing channels have on generating sales?
• What does the purchase lifecycle look like over multiple visits and how can I optimise it?
• How should I be segmenting my audience or customers, to improve the effectiveness of my marketing activity?

To answer these types of questions we are going to have to start to organise the data in different ways and we need to bring in some different tools. First of all we need to integrate our data so that we can see different aspects of the acquisition, conversion and retention processes in one place, Secondly we need to aggregate our data so that its focuses on the visitor or customer rather than the click or the visit. Thirdly we need to cut through the noise in the data using more sophisticated analytical techniques to get at the key insights. Let me give you an example of what I mean.

We all know that different types of people come to our websites for different reasons and to do different things. If I treat everyone the same, I am being sub-optimal in my decision making about how I allocate marketing funds and about how I manage the user experience. I need to segment my audience so that I can market to these different groups more effectively. However, I can’t do that on the basis on how they behave on the website alone, I need to also understand their demographics, their intentions, their aspirations and their opinions. So I need to integrate my hard core behavioural data with profiling and attitudinal data drawn from other data sources like surveys.

Next, I am interested in the behaviour of visitors over multiple visits rather than what they do in a single visit. So I need to aggregate the data so that I have a record of the behaviour of different visitors over a period of time. Also I probably need to summarise the data and create additional attributes which describe aspects of that behaviour over time such as number of visits made, number of conversions events, types of conversion events and so on.

Finally, I need to analyse the data to identify interesting and meaningful segments of visitors. In all likelihood I will probably have quite a large and noisy dataset where I won’t be able to see the forest for all the trees. Traditional querying and reporting techniques are unlikely to be an effective method of identifying the patterns, I need to use something that will find the patterns in the data for me. In this case I decide to use cluster analysis. The cluster analysis process looks for groups of visitors in the data, where the people within the groups have something in common but what they have in common is different from group to group. What I have to do then is interpret that data to understand what it is the visitor segments have been clustered on and decide whether these are meaningful and useful segments that I can do something with. This process may yield some surprising results and enable to think about the audience in a way that I had not previously thought of them before. I may find patterns and relationships in the data that I would never have found using traditional analysis techniques.

So using data mining and predictive analytical techniques will allow organisations to unlock more value from their data but it requires a different approach to managing your data, different tools and different skills. Next time I will look at another application of data mining and predictive analytics; to understand what are the important factors are that affect someone’s propensity to buy something during the purchase lifecycle.

Till then…