Should it be red or should it be blue?

We’ve all been there. Sitting round a conference room table discussing with our colleagues about the design of the website, the flow of a particular user path or the layout of a particular page. Opinions differ on what would work best, whether the call to action button should be red or blue, square or round, flat or bevelled. We all know best, because we’re experts. Aren’t we? In some cases it may not matter how expert we are, because the loudest voice will win or the most important person’s opinion will be the one that counts.

This is what happens in the absence of good data to guide decision making. It becomes subjective and vulnerable to interests, politics and personal preference. Good optimisation strategies are built on good data, coupled with an ability to drive through change. In the ‘red versus blue’ debate, what better way to find out than to try both of them and see which one works best? That’s the basis of a testing and experimentation programme. A/B testing and its more sophisticated cousin, multivariate testing or MVT, have been around for years as techniques in direct marketing and other analytical disciplines. These techniques allow you to test different versions of a page or different combinations of different elements on a page to see which one works best. Although the analytical techniques have been around for years, it is only relatively recently that they have begun to get adopted and gain traction in online marketing.

A/B testing is conceptually very simple but can be difficult to execute. To see whether the blue button would be better than the existing red button, two different versions of the page would be created, one with each different button. As visitors arrive on the website one set of visitors would be shown version A, and the remainder would be shown version B. The effectiveness of each page is measured according to whatever success criteria are the most relevant. Multivariate testing uses a more sophisticated set of algorithms to allow you to test multiple versions of different elements of a page at the same time.

Whichever testing regime is the right one, the challenges historically have been in implementation. It can be technically challenging to manage the process of setting up different versions of pages, splitting the traffic between them, setting up the measurement and then analysing the results. This heavy lifting has now largely been addressed by the availability of specialist testing and experimentation platforms such as Omniture’s Test & Target, Autonomy’s Optimost or Google’s Website Optimiser. Whilst the implementation of these services is not trivial, they manage the whole testing and experimentation process from deployment through to results.

The challenge for organisations then, still remains “What to test” and “How to test”. Analytics, survey data and usability evaluations can all help to identify the priorities for a testing programme. Start at the greatest pain points but also sort out the operational processes around the initial deployment on pages that are not too complex or mission critical. Marketing landing pages are often a good place to start. The question then becomes how should the page be tested? The challenge here is to ensure that you come up with a good test design. In our ‘red versus blue’ debate, are these the best options to test? What about green or orange? You could test those as well but in reality the number of variants that can be tested is limited. It’s important to ensure that you’re testing the right things in the right way. For me, this is where user experience expertise adds value to an analytical testing programme. Customer experience experts can identify which variants are likely to be the most effective and ensure that the test is as efficient and as effective as possible and the analytics experts can run the experiments and do the analysis.

Testing is a powerful tool in the site optimisation toolkit but it’s important to test the right things in the right way.