Surveys

This article, written by Neil Mason, was originally published on Clickz.com and is republished here with permission.

In my previous article we looked at the need to combine detailed data from site analysis systems with additional consumer insight gleaned from surveys. This week, some thoughts on how to ensure your surveys are as effective as possible.

First of all, there are many different forms of surveys that an online business might run and they can vary on a number of dimensions. For example you might be surveying visitors on your site as opposed to customers, you might be collecting some general background information or you might be asking about a specific issue. The survey might be a once off survey or it may be run on a continuous basis. Indeed some of the different dimensions that might be involved and considered in the development of an online survey include:

• The purpose of the survey
• The target audience for the survey
• The type of survey and how the respondents are recruited
• The number of responses needed
• The expected response rate
• The purpose of the survey

Purpose of the survey

Any survey should have clear objectives; there must be a reason why you want to do it. There could be more than one research objective in a survey, but it is important that they are clearly stated, easily understood and are not contradictory. From the objectives everything else flows, ie the type of survey needed, the target audience and so on.

The target audience for the survey

It should be apparent from the objectives who you want to talk to in the survey. You may not want to invest time and effort understanding everything about everybody who visits your site. Your primary interest will be about finding out the right information about the types of visitors who are of most interest to you, like customers, subscribers and so on.

The type of survey

Having determined what the survey’s objectives are and who you want to survey, you are in a better position to decide on the type of survey that is most likely to meet your needs

On the whole, there are two main types of online surveys:

• Pop-up surveys
• Site based surveys

Pop up surveys (as the name implies) pop up in a window on your site. They must generally be short and easy to answer. Site based surveys are potentially more extensive surveys that people are directed to on a separate part of the site or on another site altogether. Some of the key differences between pop-up surveys and site-based surveys are highlighted below.

The number of responses needed

Another key consideration for your survey is the number of completed responses you need. This can vary enormously with the type of work you are carrying out and the target audience for the survey itself. In general terms for consumer analysis, you would ideally be looking for about 400 respondents to allow you to be able to do any meaningful analysis.

Response rates

Having determined how many respondents you think you need, you then need to think about how you are going to get them. For a pop-up survey, visitors are typically randomly selected on the site and presented with the pop-up survey invitation. For a site-based survey, people will either be invited by e-mail or via an invitation on the site.

In either case, only a proportion of those who are invited to participate in the survey will actually do so and complete it. This proportion is known as the response rate. This response rate can vary from survey to survey and it has been found that response rates to surveys are influenced by:

• The style and quality of the survey’s first page
• Relationship with the web site and/or the brand
• The level of interest and relevance of the survey to the potential respondent

If you are inviting people to participate using an e-mail, then the style of the e-mail and the subject line will also be an important factor affecting the response rate. You should try and make the call to action as interesting and as engaging as possible so that it cuts through the noise in their Inbox. You should use language be appropriate to the type of business that you are and also the relationship that you have with the potential respondent. Many e-mail and survey systems allow you to personalise the invite and this can be used to good effect to improve the chances that someone opens the e-mail and then acts on it.

Survey frequency

The final consideration will be on how often you are going to run the survey. A great many surveys are only run once to get some insight into a particular issue, eg the effects of a new site design, but some surveys such as a customer satisfaction monitor might be run more than once or on a continuous basis.

This article, written by Neil Mason, was originally published on Clickz.com and is republished here with permission.

During the heady days of the dot-com boom in the late 1990s and early 2000s there was a common perception that this “new media” meant an end to many traditional marketing practices and I remember being told that as far as market research was concerned, that things were different in the “Internet world”.

Certainly, a more interactive experience for many businesses with their customers means that they had much more access to information on what their customers were doing (or not doing) than ever before. The web was perceived to be totally measurable through analysing behaviour on the site and that the use of traditional practices like market research was no longer appropriate to doing business in the digital age.

As we all know enormous amounts of data can be generated from online businesses about what visitors and customers are doing. It is possible to track their every movement through the website, you can tell how many people put something in their shopping basket and then either take it out again or fail to complete the check out process. It’s also possible to track exactly which creative of which ad was clicked on and which site it was clicked on to attract them to the site in the first place. We also all know of the challenges involved is generating meaningful insight from the vast quantities of web analytics data.

However, whilst site-centric data is very good at telling you what is happening it is generally very poor at telling you why it is happening. In addition, web analytics data can tell you all about what has happened in the past but doesn’t necessarily help you understand what might happen in the future.

Relying totally on analysis of web analytics data can be likened to diving down a motorway at full speed but only looking in your rear-view mirror. Whilst you can tell where you’ve been, you can’t tell what’s about to happen. Getting beyond the “what” and more into the “why” enables you to get beyond taking a purely historical perspective on the business and to form a view on where the business is going. To do this you need to get under the skin of your users and customers and understand why they do what they do and how they feel about it.

Surveys are one of the most common methods of understanding what customers think and how they feel and developments in the usability and affordability of online research tools means that conducting research amongst the customer base is now easier than ever before. However, there is a huge difference between deploying surveys that generate useful insight and pulling together a few questions or doing a poll on a website.

Concerns that have existed in the general market research world about possible biases by conducting research using online methods are not relevant to the online business. If a customer is doing business online then they are generally likely to be able to be researched using online methods.

Conducting research online can have many advantages over the more traditional approaches such as using face to face interviews or telephone research:

First of all, costs can be dramatically reduced as there is a much lower cost associated with actually collecting the data in the first place. Typically the actual costs of just collecting the data can be 50% of the overall costs of a market research study using face-to-face or telephone data collection.

Secondly, project times can be reduced. With some of the tools available for conducting online research it’s possible to write your questionnaire into a system and have it “in the field” within a day or so.

However, just because online research can be cheaper and faster, it doesn’t mean that it doesn’t deserve the same kind of rigour in its design. A badly designed survey is still a badly designed survey and it doesn’t matter whether the data is being collected face to face, over the telephone or on the web.

A badly designed survey can not only have an impact on response rates but can also have an impact on people’s perceptions of you as a brand. So whilst the increased accessibility of online research means that potentially many more businesses can use surveys as part of the marketing intelligence tool kit, they still need to ensure that those surveys have some degree of expertise applied to them as well.

Next time, some thoughts on how to improve the effectiveness of your online surveys.