In the first part I introduce the concepts of data mining and predictive analytics and present an overview of the data mining process.

Data Discovery: Tell Me Something I Don’t Know by Neil Mason – Part I

In the second part, I talk through some of the challenges in extracting and transforming web data into datasets that are suitable for data mining and predictive analytics techniques. I also introduce some of the applications of these techniques including the use of data mining approaches for understanding visitor segmentation using integrated web analytics, survey and customer data.

Data Discovery: Tell Me Something I Don’t Know by Neil Mason – Part II

In the final part of the presentation I show how these segmentation techniques can be used to identify the most valuable users of websites and how they can be used to build compelling personas. I also look at the use of techniques such as propensity modelling to understand the factors that drive different types of multi-channel buying behaviour.

Data Discovery: Tell Me Something I Don’t Know by Neil Mason – Part III

Hopefully from the presentation you get a sense of some of the opportunities to get real insight from your digital data by applying some of these analytical techniques. The process is not without its challenges so if you would like some further details, do get in touch.

Even if you’re not a Facebook addict or regular Twitter user, you’ll know how difficult it is to escape social media. Why? Because social media is revolutionising the way that people consume content.

Social media is opening new channels of communication between brands and customers and there is a lot of potential in the social web that marketers can tap into. For example, a study earlier in the year by Penn State University showed that 20% of all tweets mentioned a brand name. Sales and marketing professionals need to be aware of these significant media consumption trends so they can tailor and target their messages as effectively as possible across a changing landscape.

However, with all the hype around social media it can be difficult to understand where to start. So here is an outline plan of action to dip your toes into the world of social media:

• Set clear aims and objectives.
• Listen to the social world; understand what is going on out there and find out who your audience is.
• Use all the information you have at your fingertips to build up a picture of the social landscape.
• This information will show you how to work with your audience to achieve your goals.
• Then simply measure, refine, repeat.

More businesses should focus on developing social media campaigns relevant to their customers. The days of pure brand ‘broadcasting’ are long behind us. Successful campaigns are now being supported by an online social media component, or taking place exclusively in social media.

As you get to know more about your customers and their social media presence, you will develop a strategy that not only enables you to influence conversations about your brand and win more brand advocates, it will also recruit them as willing foot soldiers in your battle for brand supremacy.

Note: *A Report detailing this subject was written by Jonathan Culling and Billie Andersen for Evaluation Centre. To read this article please go to the crm software, contact centre software and marketing software section of the Evaluation Centre.

In the previous two columns I have been looking at different types of segmentation strategies, mainly dealing with what segmentation is, the different types of segmentation strategies and the role each type can play in building up a core understanding of your customers or prospective customers. So once you’ve decided what to create the segments on, the question then becomes about how to create the segments. Remember with segmentation what we are trying to do is to create groups of people who have something in common.

When it comes to creating segmentations, there are two main alternative approaches:

• Deterministic segmentation strategies
• Discovery segmentation strategies

With deterministic segmentations, the user or customer segments are based on some kind of hypothesis and then the data is analysed to see whether the segments are interesting and useful. For example, demographic segmentations often tend to be “deterministic”. You may segment your customers on the basis of gender and age in the belief the criteria are useful and interesting. Also most segmentation on web data that’s done at the moment is done this way. Most of the web analytics tools that people are using have some kind of segmentation capabilities built into them, allowing you to start to create hypotheses about what might be useful segments to analyse, understand and track. For example you might be interested in looking at the differences in behaviours based on the number of times people visited the site, or the channel they came in on, or the search terms used. Deterministic approaches can be successful but they can also involve a lot of time in analysis, particularly when dealing with large and complex data sets. Many iterations might be required in order to indentify segments that are meaningful, interesting and useful. This is where the power and functionality of your analytics tools becomes vitally important. If it takes you ages to create a segment and to see the results, then this will inevitably mean that you won’t arrive at an optimal solution.

Discovery based segmentation approaches use statistical and data mining algorithms to look for differences in user behaviour. Typical methodologies used here in segmentation studies include cluster analysis, neural networks and decision trees. Methods such as cluster analysis look for statistically meaningful differences between different users groups based on the data that fed into the analysis process. This is a massive area as there are many different types of segmentation techniques. Even when talking about cluster analysis, there are many different variants of cluster analysis such as k-means, hierarchical, two-step and so on. Each approach has its strengths and weaknesses and even within a single variant of cluster analysis there are many different ways that the analysis can be run. Although these solutions can be viewed as very technical, they is as much analytical “art” behind a successful outcome as there is “science”. Once those groups have been determined further analysis is done to profile the groups to understand what those differences are and whether they are meaningful or not. Just because something is statistically significant, it doesn’t mean that it is necessarily commercially significant!

Discovery based methods can yield user segments that may not be immediately obvious from the data. This is one of the benefits of using this type of approach. Quite often in the work that we have done using these types of techniques on web data we find that some of the more interesting and valuable segments are quite small, and this is because web analytics data typically contains a lot of noise from people who only ever visit the site once or twice and do nothing of any value. However, discovery based approaches require specialist skills and are highly iterative and consequently are more likely to be more costly in terms of both time and money.

Facebook turned six recently and celebrated the milestone by giving its homepage yet another makeover, this time to “improve navigation to and discovery of commonly used features”. Six years is a long time on the interweb but, even still, Facebook has made impressive and significant gains in that time. It currently sits at number four on the list of biggest names on the web (behind Google, Microsoft and Yahoo, respectively) and with over 350 million users (and growing fast) it is poised to very soon become number three. It’s become the “face”, as it were, of the social media space, if not the brain. Facebook is sitting on a veritable gold mine of highly detailed and often intensely personal information about its users; things they would never dream of telling anyone but their closest confidants, let alone a market researcher, making Facebook a veritable marketer’s wet dream. But much like war, “woah woah woah, what is it good for?” What do users actually “use” Facebook for?

Marketers claim to be reaping the benefits, as evidenced by the explosion of ads in Facebook recently, but is it to users’ benefit or their detriment? What are We getting out of the bargain? Is it enriching our lives or just making it easier to sell us stuff? Where social media succeeds over other media is by convincing its users to do the advertiser’s work for them. Every time we click a link or join a group, we might think we’re engaging in solidarity (support single sex marriage) or activism (RATM for Xmas No.1) or showing off our pop culture cred (Invader Zim), but what we are really doing is posting ads in our friends’ news feeds and sharing ads with them. As Facebook UK’s Commercial Director, Stephen Haines, puts it: [P]otential customers can directly engage with your business by clicking on the “Become a Fan” link or the “RSVP to this Event” link…this action automatically creates a story on the person’s profile page and possibly in their friends’ home page “Highlights”, generating free distribution for you.” Facebook “fan” pages are never going to replace a corporate presence on the web, but social media offers an interesting way for marketers to gather data on users’ opinions, behaviours and preferences by allowing them to consciously identify themselves with brands they like without ever feeling like a corporate stooge. And chances are they’re spending a flip of a lot more time on Facebook than they are on the corporate site. Marketers know this and they get it, but I’m not sure users do. The distinction for users is that they’re not part of a brand, they’re part of a community which identifies with that brand…but for all intents and purposes it’s an online focus group.

Similarly, market researchers are getting in on the action with Facebook Polling. Next time you click on one of those seemingly innoculous polls at your friend’s behest, thinking you’ll get to see how closely your opinion ranks against your cyber-buddies, you’ll effectively be clicking on a banner ad. You’ll also be giving the marketer who paid USD$50 to set it up the poll a candid view of your opinions, behaviours and preferences in context with the 99 other people who clicked it in your geographic region. Ray Pointer of Virtual Surveys told utalkmarketing.com: “These polls are clearly not going to replace U&A or ad-trackers, but they could spawn new ways of working. Traditionally, we have expected everything to be designed before the research begins, but often the basic assumptions were wrong.” Any one else reminded of the opening lines of HG Well’s War of the Worlds?

“No one would have believed in the last years of the nineteenth century that this world was being watched keenly and closely by intelligences greater than man’s and yet as mortal as his own; that as men busied themselves about their various concerns they were scrutinised and studied, perhaps almost as narrowly as a man with a microscope might scrutinise the transient creatures that swarm and multiply in a drop of water.”

I wouldn’t go so far as admit to the intelligectual superiority of our marketing bretheren, but they sure are some clever people.

This post originally appeared on Applied Insights’ blog. Foviance acquired Applied Insights in November 2008, with Neil Mason joining us as Director of Analytical Consulting. As part of this acquisition, we’ve incorporated Applied Insights’ blog into our own.

When we think about how to evaluate a predictive model the first thing we typically think of is how accurately does that model predict against the (unseen) test data. More often than not though when we develop models our business/research customers want more than that. They want to know how the algorithm got to the predictions i.e. they want to understand the model.

The more transparent predictive methods don’t just predict they also reveal the patterns that underlie them. The two main benefits of this are that

1. Subject Matter Experts (SMEs) typically on the business/research side – can assess the model’s validity by viewing these patterns, for example as rules or formulae. This way they can see if the inherent relationships make sense. Do they see any potential anomalies in the data that we didn’t pick up when we previously explored it?
2. And of course the patterns themselves may reveal useful insights. We often find specific segments of interest; demographic groups who have a higher propensity to convert through a given channel, or re-purchasers who have short, but potentially interesting and valuable, buying cycles.

The bottom line is that when we can see what a model is doing we can glean much more from it than the likelihood that the outcome of interest (convert, attrite, default, etc.) will happen.

To be frank most of our projects are like this. This is where Decision Tree methods often win out because the output let’s us visually explore the data to both understand the model and to examine other potential patterns of interest. They may not necessarily give us the most accurate predictions but often the SMEs care more about understanding than predicting. This is a classic trade-off in PA.

There are exceptions to this. The alternative view is that accuracy is paramount and it could be that the winning model is opaque. Neural Network models are a case in point. Depending on the software you are using you might see a ranked list of fields which contribute to the prediction along with the prediction itself and perhaps an associated confidence level. Even if the final network is displayed it doesn’t necessarily explain much more.

For the most part these are the two most typical scenarios however we are currently designing a 3^rd type – where opaqueness is the main objective (together with an acceptable level of predictive accuracy of course). We’re talking to a government department who don’t want to have to send sensitive data out and who don’t want our models to reveal any of that information either. So the gist of our approach is that we’ll develop black-box models on our data and let them deploy them on their database. They’ll give us addresses and predictive scores in return but in so doing we won’t know why a particular address was selected.

Anyone living in the UK will understand the political backdrop to this as there have been various high profile cases of data going AWOL (here is the latest one). We are hoping that a somewhat unorthodox application of Predictive Analytics might help the UK government provide a valuable public service without further compromising the confidentiality of its citizens. There’s many a slip twixt the cup and the lip mind you – we’ll keep you posted…

This post originally appeared in Applied Insights’ events section. Foviance acquired Applied Insights in November 2008, with Neil Mason joining us as Director of Analytical Consulting. As part of this acquisition, we’ve incorporated Applied Insights’ events list into our own.

Applied Insights ran a one day workshop in Predictive Analytics in association with the Emetrics Marketing Optimisation summit on 22^nd May at the Hotel Russell in London. A course outline is below.

Please contact us if you would be interested in joining one of our courses or developing a customised in-house training session on predictive analytics.

Predictive Analytics – course outline

An Introduction to Data Mining and Predictive Analytics is a one day workshop covering the foundations of this innovation marketing analytics discipline. During the course of the day you will gain a thorough familiarisation with some of the key principles and methodologies of data mining and predictive analytics and learn how to apply them to common marketing problems such as:

• How can I predict campaign response?
• How do I segment my website visitors or customers?
• How can I anticipate possible customer defections?

In this one day interactive course we will cover the following topics:

Introduction:

• What is data mining and how is that different to predictive analytics?
• How organisations are currently using data mining and predictive analytics across their businesses and to solve particular marketing problems

Processes and implementation

• How to go about a data mining/predictive analytics project
• An overview of a standard industry process (CRISP-DM)

Methods and applications

• An overview of the main types of data mining and predictive analytics applications:
  • Forecasting
  • Segmentation
  • Classification
• An introduction to main methodologies such as:
  • Time-series forecasting
  • Regression analysis
  • Decision trees (CHAID, CART and so on)
  • Cluster analysis
  • Neural networks
• Case studies and examples of how these techniques are used and deployed in both online and offline marketing is areas such as:
  • Retention modelling
  • Conversion propensity modelling
  • Visitor segmentation

This post originally appeared in Applied Insights’ events section. Foviance acquired Applied Insights in November 2008, with Neil Mason joining us as Director of Analytical Consulting. As part of this acquisition, we’ve incorporated Applied Insights’ events list into our own.

At this year’s annual Web Analytics Congress in Holland, Neil delivered a keynote presentation on Marketing Optimisation and Predictive Analytics.

This post originally appeared in Applied Insights’ events section. Foviance acquired Applied Insights in November 2008, with Neil Mason joining us as Director of Analytical Consulting. As part of this acquisition, we’ve incorporated Applied Insights’ events list into our own.

At this year’s Emetrics Summit in San Fransisco, Neil will be presenting a session in the “Advanced Analytics Track” entitled ‘Cutting through the NOISE: Applications of data mining and predictive analytics’.

The presentation will be looking at the application of techniques such as segmentation and propensity modelling to better understand website visitor behaviour.

This post originally appeared in Applied Insights’ events section. Foviance acquired Applied Insights in November 2008, with Neil Mason joining us as Director of Analytical Consulting. As part of this acquisition, we’ve incorporated Applied Insights’ events list into our own.

A return visit to Stockholm by Applied Insights this year. This time we’ll be giving a presentation at the Internet Marketing Conference on “Predictive Analytics – Why Bother?”. We’ve also been asked to be on the panel on the subject of Testing and Analysis and have been roped in to moderating a panel session on Web Analytics. Should be interesting…

This post originally appeared in Applied Insights’ events section. Foviance acquired Applied Insights in November 2008, with Neil Mason joining us as Director of Analytical Consulting. As part of this acquisition, we’ve incorporated Applied Insights’ events list into our own.

At this year’s Emetrics Summit in Washington DC, Neil presented a paper entitled “Cutting through the NOISE: Applications of data mining and predictive analytics”.