Understand Your Customers with AB Testing

Now imagine doing any of these things on a smartphone. Chances are it would take several false starts and -- if you were dedicated to the goal -- a lot of struggle before the job was done. It has little if anything to do with the screen size. Instead, it’s a reflection of design and content that doesn't match the contextual needs of the user. And it doesn't take an extreme example to illustrate this point.

For all of the attention it has received, the mobile web remains a frustrating place for users. Businesses big and small, across industries, struggle to create seamless, functional and appropriate experiences for the many channels through which consumers now engage. At the same time, mobile optimized websites are increasingly important for businesses.

In January of 2014, mobile app and web use eclipsed the desktop in terms of total time spent with the Internet for the first time ever. Underscoring this trend, in a recent survey conducted by Braun Research, 91 percent of millennial respondents claimed their smartphones were just as important as their cars -- or deodorant. Clearly, to communicate with this growing group of consumers -- and consumers in general -- businesses must provide an effective, enjoyable mobile experience.

Through extensive usability testing, the Nielsen Norman Group has identified four important elements for the design of cross channel experiences.

  • Consistency, established and maintained through a persistent visual design and tone of voice, helps orient users within each channel -- even on the first visit via an unfamiliar device.
  • Creating a seamless experience is also important -- one that allows users to start a task on one device and finish it on another.
  • Availability of features across channels and devices is critical.
  • And perhaps most importantly, experiences must be context specific -- prioritizing information and functionality in accordance with the needs of users within the most likely environments each device will be used.

As important as these elements are, most can be addressed with a careful design and copywriting strategy. However, the last factor -- creating an appropriate, context-specific experience -- is more challenging. And since it addresses the needs, intent and unique mentality of users, it’s also the most important. Creating a context-specific experience requires more than design expertise or an understanding of industry best practices. Instead, each business must determine the needs of their own customers for each channel.

A/B testing and optimization are the most powerful tools available for building this understanding of actual customers. Through an iterative, experimental approach, it’s possible to create detailed user personas for each channel and, more importantly, verify strategies for addressing their needs and desires.

Specifically, testing can explore these three critical factors:

1. Establishing Authentic Relevancy

Thanks to engineering frameworks like responsive design, it’s easy to create a website that stretches and shrinks to fit the frame of nearly any device. Such an approach solves the problems of consistency and possibly seamlessness discussed above, but fails to address needs for context specificity. Mobile users, for example, may need simplified or even different information. Tablet users may respond to different images than desktop users. Establishing this relevancy can be difficult.

Traditionally, surveys and focus groups would have been necessary to understand what users relate to in each context. A/B testing can uncover much of the same information -- and even more, by avoiding experimenter influence and participant bias -- quickly. Testing multiple headlines, for example, is as simple as a few mouse clicks in most testing platforms. Tests that target certain segments with tailored language, images, and designs can be more complex but yield actionable insights that can drive omnichannel strategy.

2. Understanding the Path to Purchase

In the physical world, different products have different paths to purchase -- buying a new car, for example, is usually a longer process than buying an ice cream cone. This is obviously true online as well. Interesting research conducted by Google and Nielsen has found that the context influences the path to purchase as well. In the mobile world, this research suggests, the path to purchase is both short and active -- though research is still important, 83 percent of consumers studied completed purchases within one day and 55 percent completed the process within one hour.

Testing can help understand this factor as well. An e-commerce retailer could compare the performance of checkout opportunities placed at various levels of the site and to differing degrees of prominence. Software-as-service companies could test subscribe call-to-action buttons in a similar way.

3. Aligning with Customer Needs

If the sales cycle is understood as a funnel, there’s no need to create unique strategies for each device. Simply stuff the top of the funnel and a few motivated users will eventually drip out the bottom. Unfortunately, this model doesn’t accurately reflect user behavior. Instead, consumers move through a journey, defined by multiple touch points, that eventually lead to the purchase decision.

By testing different content -- focused on research and comparison or purchase options -- on various devices, it’s possible to align each site with the specific needs of customers utilizing that channel. Using these insights to develop a map of the consumer decision journey can help brands create an omnichannel experience that more effectively appeals to customers and users within each context.

As shopping has spread from physical stores to e-commerce websites, desktops to mobile devices, a need for unified, seamless experiences has emerged. Relevancy, an appropriate path to purchase and alignment to specific user needs are all critical factors for optimizing omnichannel experiences. Though no “one size fits all” solution meets these requirements, A/B testing can help determine the best strategy -- and deepen the relationship between brands and their audiences.

Title image by marfis75 (Flickr) via a CC BY-SA 2.0 license