Mobile Usability Testing for 2010 and Beyond

By Jason Munson (@JasonMunson) Sep 28, 2010

As the mobile channel’s audience has progressed from early adopters to the mainstream, many businesses are now providing mobile experiences for their users. The shifting momentum from desktop experiences to mobile has forced many online managers to stretch themselves into understanding this additional channel’s nuances and how to design mobile optimized user experiences (UX). Here's a look at how Usability Testing can help and how it's different from testing for the desktop.

Once best practices knowledge is leveraged to design your first mobile website, every Customer Experience (CX) obsessed professional will want to test, learn and refine. Analytics can help identify the area(s) where Usability Testing is needed, which is a familiar concept and process for most.

Similarly, most CX and UX focused professionals have a clear notion of what a Usability Test for a desktop website looks, sounds and feels like. Whether the testing is conducted in the lab, field or via remote desktop connection, the stakeholders have a good sense of what to look for and how to make the needed improvements.

(Also by Jason Munson: Enterprise 2.0: Increase Productivity and Extend Reach via Mobile)

There are similarities in measuring and testing experiences across the desktop and mobile channels, but there are also some new perspectives to be considered. In today’s post I will touch on two of the big decision points that you will need to consider when you tackle mobile usability testing: Identifying the Devices you will test, and the Testing Methods and Environment.

Identifying the Devices

When running a usability test for mobile devices, one of the first things to consider should be the devices you are going to test. This can be a significant challenge, as the device pool is constantly evolving with newer-cooler-sleeker products hitting the market on a weekly basis.

The obvious first step is to know your audience, and consult your analytics to get a sense of the devices that are being used to access your existing web property. Even if your project is the first mobile channel experience your brand is releasing to market, knowing who’s tried to interact with your site via mobile is certainly important.

Once you know what the device pool looks like, you’ll either need to make a determination on the specific devices you want to test (Apple iPhone 4, Motorola Droid 2, Dell Streak, etc.) and consider things like screen size, screen layout and interaction style… or … define and choose device categories for testing.

Testing Device Categories

Testing device categories is a good option for companies that have decided to cater the experience to a more granular degree than “one size fits-all” (a.k.a the Lowest Common Denominator) but are not optimizing on a device specific level. If this sounds like a sensible approach, you’ll want to make sure you make this decision starting at the design phase, to achieve optimal results.

Some examples of possible device categories are:

Touch-Screen Mobile Phones (Apple iPhones, HTC Nexus One, HTC EVO, etc.)
Non Touch-Screen Mobile Phones (RIM Blackberry Bold, Motorola Droid 2, etc.)
Quick Messaging Devices (LG Xenon, Samsung Intensity, etc.)
Tablets (Apple iPad, Samsung Galaxy, Dell Streak, etc.)

Testing Methods and Environment

The existence of statistically meaningful differences in test results conducted on mobile user interfaces under different environments remains a topic of debate for many.