User Testing

July 30, 2009 – Foviance, the experts in customer experience, were appointed in 4Q 2008 by Turner Broadcasting to evaluate the user experience of their newly developed Cartoon Network website launching across 12 EMEA territories in 2009.Research was conducted by Foviance in the UK, Poland and Spain, working in collaboration with Xperience Consulting in Spain and UseLab in Poland. This research played a key part in the user centred development of Cartoon Network’s new website, which aims to increase the level of kids’ engagement, entertainment and reward. The UK website has recently launched as part of the EMEA wide rollout. Read more…

At Foviance, our job is to help the world’s leading brands understand their customers better. That means uncovering who they are, along with their expectations, goals, behaviours and attitudes. We use this knowledge to deliver an experience that pleases end customers while ensuring commercial success for our clients. Read more…

We’ve all been there. Sitting round a conference room table discussing with our colleagues about the design of the website, the flow of a particular user path or the layout of a particular page. Opinions differ on what would work best, whether the call to action button should be red or blue, square or round, flat or bevelled. We all know best, because we’re experts. Aren’t we? In some cases it may not matter how expert we are, because the loudest voice will win or the most important person’s opinion will be the one that counts. Read more…

By Amanda Roach and Catherine Fox

Ethnography is a research technique used to inform intuitive design by capturing information about people’s behaviour in the context of their own environments. It is proven to be an effective method of gaining accurate insights into customer experience and is particularly effective method at the beginning of a project when no assumptions have been made about potential findings.

At Foviance we like to think that ethnography allows us to walk in our customers’ shoes, not just observing what they do, but also what they don’t do, and what they don’t realise they are doing. In a business environment, for example, many people perform numerous physical and social actions during the day that are not strictly defined by their working role. Individuals develop workarounds, they work with colleagues on problematic tasks, and even establish consensual best practices for dealing with certain issues – all often without realising what they are really doing. Indeed if you actually ask people to describe what they do during an average working day, they’ll only think to tell you of around 20 percent of their activities.

Ethnography is most commonly associated with immersive observation, but can be employed to great effect at all stages of the customer experience cycle. It can be used to discover the ways by which people reach purchasing decisions, how they browse in shops, how they flick through catalogues, how they deal with the out-of-box experience of a new product, how they interact with different interfaces, how they handle glitches or problems, and of course the journeys they take through websites. Technology enables us to look at the totality of the user experience, joining up observations and arguably moving beyond the Margaret Mead approach to anthropology and sociology, by adding engaged interaction to more passive observation.

We’ve used ethnography worldwide to measure experiences for global brands in a variety of regions. One ethnographic project saw us working for a major computer retailer in China. The company did not want to make assumptions about the work environments of people employed in both small and large businesses. We entered these environments and measured dimensions in their workspace, observed working conditions, examined how individual workers set up their workstations, evaluated noise levels, noted the locations of windows, measured co-worker interaction and more.

However, we have also been tasked with measuring individual learning curves with new mobile phone handsets, and have achieved this somewhat differently by allowing participants to use the phones in a natural way in their own environments, keeping a diary of experiences for further discussion. We were even able to ask individuals to take photos with their handsets at different times during their day to record their environment and experiences, enhancing the verbal analysis with a visual record and a real sense of the mobile experience.

These ‘digital ethnography’ techniques have to some extent allowed us to combat the ‘Hawthorne effect’, a studied reaction that can cause subjects to alter their measurable behaviours simply because they are being observed. The closer we can get to allowing participants in research to evaluate products and services in their own real-world environment, the more likely we can avoid such problems. Even sending consultants into working environments can prove problematic, with employees worrying that their bosses might be critical, for example, of their workarounds. In this sense, the role of the ethnographer is to disappear into woodwork, to attempt to put people at ease, and to use probing questions.

Sometimes we get to go to people’s homes and see for ourselves how they grab time to use the services we are evaluating between their chores and home life. It can be a real eye-opener and provides powerful insights for clients looking to develop their services to fit better into their customers’ real lives.

Ethnography is a broad discipline. The most appropriate applications of ethnography must be used in the right circumstances, and more often than not they will also be used in conjunction with other methodologies such as lab-based testing and user groups. Often we will observe a working structure or a product being used and simply report. But other projects may be highly contextual – for example we have discovered keyboards that were regarded highly in a laboratory environment but were too reflective to use outside in bright sunlight!

This is why we try to put realism into the experience, so we can feel and see potential end customer pain points – these are real people in real contexts.

 By Clare Mitchell Crow

I attended the eConsultancy user and customer experience roundtable last week, where I was able share knowledge and experience about the current state of play online, and listen to stories from others working in the field.

I was heartened to hear from the majority of those attendees working in house, their plans for 2009 are largely intact, as budgets concerning customer experience are still available – this is in line with a recent US based article I read from Forrester. And it is not surprising in the current climate, that for some, measurements are moving more to customer engagement, rather than simply conversion.

Multi-variant testing was discussed, as some attendees currently use the technology; while others appeared sceptical of investment (financial outlay, design, technical) – they are more interested in placing their efforts into refining concepts and copy as a part of an iterative design process, engaging with customers along the way.

Lastly,  the discussion I valued the most was on measuring customer experience using more than just the online channel. For example, looking at channel management from call centre to web, click to call functions, and the total cost of a sale. Here I got the feeling that the industry is moving in the right direction, but there are few organisations out there who have really taken the time to define comprehensive channel management programs. At the moment the majority of approaches are of a more tactical nature, focusing on particular products. Let’s hope that with maturity, will come broader approaches.

By Lucy Carruthers

Chris Lake of Econsultancy recently published a blog called 50 swinish ways to annoy web users. It’s an amusing read, but what struck me is that the first 7 items related to forms of advertising on the web (Auto-play, interstitials, death by Adsense, death by banners, pop-ups, etc.)Advertising on web sites is an interesting conundrum. The participants in our studies are often quite vocal in their annoyance of them, yet online advertising generates big money for companies so we must be clicking on them. In fact, the IAB have recently released a press release stating that in the US Internet advertising revenues surpassed $23 Billion in 2008, reaching a record high. So like them or loathe them, adverts are a big part of our online world.

Our clients often ask how best to approach advertising on their site. Is brighter and bolder better? Should adverts be contextual and personalised? Should they wait until the end of a customer journey to upsell or is that too late? How can they create a design that provides the best balance between increasing their revenue without infuriating their customers at the risk of damaging their revenue?

I recently tested a high-fidelity black and white prototype for a financial client that contained placeholders for adverts. On pages containing non-personal content, the advertising wasn’t questioned by the participants as it appeared relevant to the main content, and also offered potential savings and discounts to the users. The participants were less keen to see similar material on pages where the main content was personal to them; however, their attitude was that as long as they weren’t distracted from their tasks they weren’t too bothered. But, as soon as the same participants saw the same pages but with the graphic design overlaid their attitude immediately changed; they were enraged to see ‘advertising’ amongst their personal account information despite the savings promoted. The graphic design had made the adverts far more prominent and as far as the participants were concerned they were intrusive and out of place.

So what can organisations do?

Firstly, organisations should see how their competitors are doing it and determine what works and what doesn’t. Online banking, for example, is an area where adverts are becoming ubiquitous and I think some banks have got their placement right, while others rudely interrupt the users journey.

Secondly, they should look for innovative new ways to advertise while keeping the users goals and needs in mind. Pixazza, a company in the States, have developed a means of converting photos on web sites into interactive advertisements. James Everingham, who is behind the concept, says that many adverts are an “obnoxious intrusion” and that “ads work best when they don’t resemble ads but rather a feature that helps shoppers find items they may already be interested in buying”.

Lastly, it’s crucial to test any designs with users and to test iteratively. Testing with early prototypes can catch key issues before it gets expensive to make changes, and testing once the final design has been overlaid can ensure that with the graphic design the site still meets the users needs and expectations. The graphic design can have a major impact on both on the usability of the site and the user experience; and satisfaction levels for the overall customer experience.

By Billie Andersen

Although remote testing will never be a complete replacement for traditional user testing, it does have its role in the user experience professional’s tool box – and it’s becoming an increasingly important one. Read more…

 By Clare Mitchell Crow

When Foviance is tasked with conducting international customer experience testing, we are usually responding to a client’s natural concerns over the different languages and linguistics, commercial practices, or aesthetic preferences customers might have in the different regions around the world in which they wish to operate commercially.

While scoping or developing interest for their brand in new markets, global companies will nearly always consider developing e-commerce before any other channel. Understandably, online payments preferences that secure sales are usually one of the very first considerations. Consumers and businesses around the world often have varying conventions when using payments systems, and you don’t have to travel far to find significant cultural differences. In the UK on the whole, we are happy to enter personal details into secure websites, and so credit cards have become the norm. A short hop to Romania though sees cash on delivery as the usual method of online payment, while in Germany bank transfers are still common place, and some of the very biggest German brands continue to run thriving catalogue stores, with their website acting as only a front door.

Something as seemingly straightforward as filling in address details can quickly provide customers in some regions with unsurpassable problems if they are expected to use generic ‘international’ websites. Many countries have no concept of postcodes or zip codes, for example, while others have developed convenient methods of putting entire addresses on a single line. If a business adopts a ‘standard’ payment screen and doesn’t see any value in localisation, it’s inevitable that it will lose a large percentage, if not all, sales from some regions. Even if this is changed at a later date, that decision to initially ignore local practices can inevitably lower trust and brand expectations for years and affect returning custom.

Of course customers in some countries actually assume that businesses will ask them for a lot more information. How would you feel if a UK mobile phone company asked for your National Insurance number, for example? It’s a common practice in Romania, and consumers in this country might even lose trust not to be asked such a level of information.

We find that another invaluable way to engender trust in potential customers is to ensure all online content is translated by someone who lives and deals with businesses in each region. A UK or US citizen may have fantastic academic Spanish, but that doesn’t mean that their skills would be appropriate in a Mexican, Puerto Rican or even Spanish sales environment. I recall one example when an excellent Polish speaker from the UK found their translation ridiculed by local Polish focus groups because of an extreme formality in their style that just wasn’t suitable for the casual nature of the website. The site required a different tone, that was understood by the target market – local Polish speakers.

When developing expansion plans, some companies are unsure whether to launch the full offering of their core site in the new markets, or to develop slimmed down versions. Will local customers be annoyed that they don’t have access to a company’s full stock, or will there be demand for more? The only answer in these circumstances is for those companies to do their research and to make sure, rather than make assumptions. Look at the uptake of social networking among children, for example. It’s not at all unusual for kids in the UK to have phones, email addresses and online community logins at ages as young as 11, but businesses cannot simply assume that this is acceptable in other countries. Whatever the market, companies must develop the right online environments for the right regions if they want to produce successful sites.

You’d be surprised to hear some of the diverse considerations people have in the many countries in which Foviance and its partners conduct experience testing. What about something as fundamental as a URL for example? Sure, consumers in some countries prefer to trust a local URL suffix, but others feel safer with a .com, while still other countries may have restrictions on registering local URLs dependent on demonstrable local investment. Nothing, not even a website address, should go without some analysis. Pictures, graphics, colours even – some are perceived to be the right fit for products in some markets but not in others. Does a red and yellow banner promise value or just look cheap?

Ultimately, international user testing is all about caring enough about local customers to establish brands soundly and develop the best environments possible in which to encourage repeat business, sales conversion and site stickiness. There’s little point getting the marketing push right if the customer experience is impractical or simply engenders the wrong reaction. Global brands encountering some of the pitfalls discussed, soon see the economic effects through analytics, site traffic numbers and sales figures. We would never recommend that even the strongest brands extend their reach until they have conducted thorough local research with a team of customer experience experts.

If you’re still in doubt, let me remind you of Travelocity’s first attempt to launch in France with no local research. When the numbers failed to add up, the company finally asked local focus groups for a reason why this might be so. It turned out that customers just couldn’t get past the ‘local’ translation of the company’s name – Travelo. French consumers were apparently put off by the idea of booking their holiday through a ‘Drag queen’…