Surveying international opinion

There are obvious attractions to conducting international surveys as conduits of quantitative research.

Not only will they extend your reach and influence geographically, but they also expand the size of your potential samples, while reducing costs considerably compared to securing a similar scope of respondents locally. Surveys do have obvious limitations against one-to-one qualitative studies, but with skilled questioning and efficient organisation, excellent results can still be gained for far lower overheads.

Of course there are also numerous challenges involved, not least the logistics of recruiting a panel provider that can operate successfully in the desired region. In most countries this isn’t a problem, but in some it does tend to take a little more time and effort – notably Indonesia, Kenya and strangely enough, Foviance’s latest locality, Singapore. Determining and recruiting the right type of respondent can be tricky. A particular type of user or segment will have differing criteria and often, screening questions must be country-specific. When recruiting segments defined by the client, screening questions should be adapted to suit the differences between countries.

Is it possible to recruit an English-speaking panel? In some countries such as USA and Australia, there shouldn’t be much of a problem, but what about countries like France? It’s certainly possible to define an English speaking panel in a non-English speaking country but this has a cost implication as these people are more difficult to recruit. This cost may be offset if there are a number of free-text entry fields as there is no necessity to have comments translated. In addition, there would be no cost for translation both for the survey script and the verbatim comments. In some regions such as India, many different languages and dialects are spoken and English is often the ‘lingua franca’ that allows all the people to have a voice. Of course, there are disadvantages of using English speakers as they would most likely have a high level of education and a higher social economic status. They may also work for a multi-national country and so be more ‘western’, or they may be urbanites (rural communities in many countries may not take part due to the unavailability of a computer).

When it is deemed necessary to use a local language for the survey, original questions must firstly be translated. This may sound easy but it can often prove problematic due to the quality and nuance of the translation. A translator must be trusted to deliver exactly what you need.

More often than not, an English person who is fluent in another language will not be able to conduct a translation in a meaningful way. They may not be aware of the everyday (as opposed to the academic) translations for certain sentences and terms. It can prove to be very off-putting providing opinions to a non-national who seemingly hasn’t bothered to ask you properly. Also, they may not be aware of the translations for technical terms such as ‘battery-life’, jacks, HDMI and so on. Also, from a more practical point of view, translations of an English sentence which fits very well into the survey design will often be too long when translated into a language such as Russian. A conversation with the translator is then required so that a better sentence can be constructed.

Screening by socioeconomic status might simply not be possible in some countries in which those people in the lower categories do not prove literate either in English or in their own language. It is always necessary to consider from the outset whether it is feasible to fulfil certain screening criteria in particular countries of interest. For instance, in areas of India, smartphone and laptop usage will not be prevalent in those aged 55+ but will be more representative (and consequently, more weighted) for those in a younger age bracket e.g. 25-34.

With experience and endeavour, excellent results can be gleaned from international surveys. I usually split survey results in four ways:  by overall results, by region, by age and by gender. I don’t generally drill down deeper than this, by region then by age for instance, unless something really interesting crops up in the data. To give you an example, in one survey I conducted there were some really strange results within the USA. On further inspection, over 70% of responses had come from those aged 65+, providing a very biased view on mobile phone usage. To restore balance we had to re-recruit those in the younger age ranges and weight the responses appropriately so that all ages were represented equally. It just goes to show the trust you have to place in your panel provider.

Organising, conducting and analysing the results from international surveys can be a time-consuming but very enlightening process. It genuinely brings to the fore the often stark differences in attitude, usually between Western and non-Western countries, towards modern devices like smartphones as well as emerging technologies.

This article was written as part of our February newsletter

Originally written by Amanda Roach