Getting to grips with global product testing

In addition to international customer experience testing for websites and ecommerce stores, Foviance is also involved with global projects that evaluate the suitability of and reactions to products across multiple regions.

Our global examination of the varying regional customer experiences with netbooks is a great example of what can be achieved, and avoided, with thorough usability testing. Netbooks are compact, portable laptop computers designed primarily to power web browsing, electronic messaging and wireless communications. They are extremely desirable objects, especially when coveted from afar, but as we discovered, the actual user experience of these devices varies wildly depending on the country the consumer comes from.

I was part of team tasked with evaluating four different brands of netbook with end users in the UK, US and Japan. We allowed each group of individuals to gain an initial impression of the devices, by providing them with the level of look and feel insight you might gain in a shop environment. Following this introduction, we then allowed our international test groups to get to grips with the netbooks so that they could evaluate usability through a series of tasks. The idea was to see how likely they were to buy a netbook before and after assessing their practicality. By the end of the testing, a clear picture emerged that on the whole, netbook usability doesn’t live up to initial impressions.

Keyboard entry proved a problem in Japan in particular, where users were completely turned off by the idea of netbooks in just an hour and a half. Of course different cultures had different impressions, depending on their country’s level of design centricity and emphasis on usability. Our Japanese testers enjoyed the modern slick looks but found the devices unusable, whereas our US group thought the devices worked okay but didn’t like the fingerprints left on the shiny surfaces.

This close-up and personal testing is logistically hard to perform. We used our usability partners to ensure we had two testers with each individual, monitoring reactions and engaging with the testers on a personal level. We needed to buy each product locally to ensure language and layout compatibility of keyboards and screens, and we had to rely on the knowledge of local agencies to recruit from the most appropriate customer segments. At all times we tried to make the testing interactive and fun, allowing testers to sit down, pick the things up, use them as they would hope to in their own lives, and role-play. We found the most effective testing embraced a combination of ethnography and lab testing, whereby testers may not have been in their own environment, but were certainly at ease.

As a result, our manufacturer clients were blown away by the results, and were stunned by the dramatic swings in opinion before and after usage – particular with regards our Japanese group. These testers simply couldn’t get used to the keyboard, which made Japanese style data entry – focused largely around the enter key – feel clumsy, slow and impractical. It was a critical issue that our manufacturers hadn’t really considered at all.

Our product testing unlocks problems arising from cultural or regional differences and makes them transparent to vendors and designers. We provide a level of expert local insight in diverse markets that manufacturers simply can’t replicate adequately themselves. With our help, clients can re-evaluate products so that they fully live up to user expectations in different countries around the world and secure the sales returns that the core technology truly deserves. Companies that invest in their customers, inevitably reap the greatest cultural and economic rewards.

Originally written by Amanda Roach