Signals from Noise

This article, written by Neil Mason, was originally published on Clickz.com on 07/05/10 and is republished here with permission.

As I write this I’ve just got back to the UK after presenting at the eMetrics Marketing Optimisation Summit in San Jose. As usual I’m trying to process all the input I’ve had over the three day conference and extract some of the key themes. With over 40 sessions running in multiple tracks over the 3 days, it was impossible to cover everything but one of the phrases that kept popping into my head over the duration of the conference was “Signals from Noise”. It’s a phrase that I used in my own presentation to highlight the challenges that we all face as businesses in processing the enormous amount of data that we have available to us at the moment. The web is a very noisy place and the data it generates and the data that we collect is also very noisy. Our role as analysts is extract signals from the noise, to interpret those signals and to produce actionable insights.

One of the things that struck me during the conference is that the “signals from noise” issue is getting worse as the data landscape gets more complex. A core thrust of the conference was around social media measurement and it’s fast becoming a discipline in its own right. Jim Sterne also talked about “signals from noise” in his opening keynote presentation on social media metrics and about the need to start to attach a value to social media activity. As the data landscape becomes more complex, different tools are needed to collect and analyse these different types of data. Avinash Kaushik used his keynote presentation to take use through a variety of different tools to help us grapple with the complexities of long tail and social media an analysis and also reminded us that the goals is to ensure that we are measuring outcomes and not just activity. Scott Calise from MTV showed us a way to develop a coherent approach to social media measurement by being clear about the business objectives and putting the right processes in place to track the impacts. Perhaps nothing radically new there but lots of people were taking notes!

This year there was also an increased emphasis on more advanced analytical techniques. I had the opportunity to talk about the use of data mining and predictive analytical techniques in the digital space in areas such as visitor segmentation and propensity modelling. Other sessions I attended dealt with issues such as simulation techniques to understand long-term trends and the thorny issue of campaign attribution. It was good to see some of these classic marketing analysis techniques gaining traction in the online world.

I was also reminded that extracting signals from noise is not just about the use of different tools or the application of more advanced analytical techniques, it can also just be about effective presentation. The session from Gary Angel from Semphonic and Russ Rueden from Kohler took us through their approach to use-case analysis in usability and the way that the information is presented internally back into Kohler. Use-case analysis provides the structure for analysis and problem solving but I was also struck by the simple and coherent presentation framework they use to communicate the results back into the organisation. It was generally one or two slides that stated the problem, provided the supporting evidence and analysis, made specific recommendations and then showed real world examples of those recommendations in action on other sites. Angel and Rueden stressed that it was this last part that was a vital part of the process as it allowed executives to visualise what the recommendations would actually mean for their organisation.

This type of information visualisation is a vital component of extorting signals from noise so it wasn’t surprising that the panel discussion on marketing dashboards was standing room only. Interestingly there wasn’t as much discussion as I might have expected about the actual technologies for dashboard production, the discussion was more about what to actually put into dashboards and then how to get people to take action as a result. The conclusion is that you need to make sure that the signals on the dashboard are the right signals and that they are pushed to the right people in the organisation in the right way. This requires the person producing the dashboard to ensure that they fully understand the business requirements for the and to then translate that into a dashboard using whatever the most appropriate technology is.

The other point that was made by the panel was that dashboards benefit enormously from added insight and that means some additional value being delivered by the analyst. This brings me back to my first point about the role of analysts is to extract signals from noise, not just to report the noise. The Web Analytics Association (WAA) held its first certification sessions at the conference and so we may well soon have our first WAA Certified Web Analyst (CWA). One very experienced analyst that I spoke to who took the test said that it certainly made her think and that the most challenging part was the case study analysis. Again, signals from noise.

Finally, I’d just like to say that it was great to meet some of you who read this column in San Jose. I appreciated your comments and feedback.