So what is Predictive Analytics?

This post originally appeared on Applied Insights’ blog. Foviance acquired Applied Insights in November 2008, with Neil Mason joining us as Director of Analytical Consulting. As part of this acquisition, we’ve incorporated Applied Insights’ blog into our own.

Over the past few years we’ve all been hearing more and more about ‘Predictive Analytics’ (PA for short). It is one of those terms – like Business Intelligence (BI), Customer Relationship Management (CRM), Enterprise Resource Planning, (ERP) etc. – that once coined, captures the essence of an area of business/research activity and gets a life of its own – particularly as software and services vendors use it as a means to more easily describe and, of course, sell their products and services.

A simple definition of predictive analytics

A simple definition of predictive analytics is that it’s an activity which allows us to quantify future events or actions. This quantification could be as straightforward as generating a list of customers who are likely, at a point in time, to behave in a certain way, e.g. to churn, to register, to buy or to respond to a particular mail, etc. Typically this list will be accompanied by a score which gives us the probability (or propensity) that an event will occur.
Alternatively the technique may give us a value, or set of values, e.g. a set of sales forecasts for a given product line in the coming week; furthermore the forecasts are often presented with confidence values, e.g. we might predict that we will sell 50 widgets on Monday but we can say, with 95% confidence, that we will sell between 45 and 55.

So what’s new?

One of the main characteristics of PA is that many of the tools, techniques and applications which it comprises are not actually particularly new. Credit scoring is one of the most well-known applications of PA. and credit scoring has been around for over 50 years.

Wikipedia details many of the other uses and techniques which can be described as ‘predictive’ though the current list there is by no means exhaustive. You will probably recognise a lot of them. In fact many of them come from the world of Data Mining and there is some significant overlap between Predictive Analytics and Data Mining, but there are also many differences; more on that later.
So is ‘predictive analytics’ just a new bottle for a lot of old wine? One of the new things is that there is an abundance of relatively new technology which can:

• Take PA to a wider audience by making the often complex algorithms more usable for less statistical/technical users.
• Provide access to a broader range of techniques through smarter user interfaces which map more closely to the analytical process in such a way that modellers (analysts, statisticians, data miners, demand forecasters, etc.) can more productively access data , test and develop models and ultimately deploy the best ones.
• Allow the results of PA (e.g. customer lists, scores, models, etc.) to be used more easily in decision making processes going forward.

For example, PA is being used to enable customer services representatives in a call centre to prevent customers churning with appropriate offers. It’s also be used in web site recommendation engines which can serve up relevant content based on what is understood of a visitor’s needs, preferences and previous browsing/buying behaviour.

This last point could mean generating models in a format (such as PMML) which can be more easily plugged-in to an operational process or integrating PA into other tools like CRM platforms.

There are also many new techniques and new applications for those techniques. The theories around Naïve Bayes and Robust Regression, for example, may have been around for a while but it is only recently that they have been available in an accessible commercial format. Techniques to automate the search for the best fitting time-series model are also quite contemporary. Applications such as SPAM detection and multivariate testing to optimise on-line marketing campaigns and page content are arguably the latest in a long line. Even if your potential application doesn’t seem to feature in any list of previous ones it could be worth exploring the potential that PA offers. There is a first time for everything!

So what can predictive analytics do for me?

PA is more commonly applied within organisations to address specific issues like preventing customer attrition or to identify segments of customers who will be more responsive to specific campaigns. One of its advantages is that it is usually possible to demonstrate potential ROI through the model, and if the model is a good one, the actual ROI when the model is deployed should not be too far from the prediction.

Increasingly though these techniques are being used more strategically to inform a range of organisational decisions and actions. Effective CRM programmes, for example, often use PA in a number of ways to anticipate customer needs and behaviours across channels and at different points in the customer lifetime.

So more sophisticated applications will not only identify good prospects for acquisition they will help define a series of interactions as the prospect develops into a loyal customer which enhance the customer experience and ultimately drive mutual benefit.

Getting more into predictive analytics

Despite the publicity I feel that applications of predictive analytics, particularly in business, are only scratching the surface today. There are a number of reasons for that; among these is the age old one that there is a gap between the available tools/techniques and the appropriate circumstances for business adoption. There is also a dearth of resource to help bridge that gap!

In future blogs I’ll explore predictive analytics in detail; looking more closely at the tools, techniques, applications and vendors. I’ll give a view on the process of applying PA – particularly aimed at those who haven’t started yet. I’ll also look beyond the hype into cases where it has been used to achieve significant results.

I’ll even discuss how PA relates to BI, CRM and ERP. Though we shall try and do that in a more understandable way than I did in that last sentence!