Editorial
As a poor college student, I would have done nearly anything for an extra buck. So when I was offered $20 to be observed while trying to accomplish various tasks on a 1990s website, I jumped at the chance.
Being young and eager to please, I worked hard to find errors and problems on the site. The observers hadasked me to vocalize my thought process,soover the course of an hour,I dredged up enough nitpicking issues to make Oscar the Grouch proud.
Finding Real Versus Invented Problems
Looking back, I realize that although manyof the problems I identified really were annoying, some of theissuesI flaggedin the lab would never have bothered me in a real-world setting. The flip side was true as well: There were probably some very real siteissues that I never noticed, since I was followingspecific instructions aboutwhat to try.
In short, my usability test did not accurately reflect what my actual user experience would have been hadthe testers left meto explore the site on my own.
The Limits of Usability Testing
This recollectiongot me tothinking recentlyabout the merits and shortcomingsof usability testing in today’smodern digital arena. Traditionally, usability testinghas been used to give usinsight into customers’ visceral and emotional reactions to branding and design, as well as their more generalized subjective responses to the holistic onsite experience.
Yet the massive scale, agility and responsiveness to design subtleties that characterize today’s e-commerce sites make it nearly impossible —and way too costly —for marketers to keep up with their customers through traditional usability testing alone.
The Power of Customer Experience Management
Today, brands in search of a truly accurate reflection of their customers’ needs and intentionsrequirea more comprehensive customer experience management(CXM) approach that letsbrands track the digital body language of customersas theyactually behave online in their“natural habitats.”
What’s more, onlyby viewing that “untamed” user experience in the aggregate can brands gain a truly accurate sense of what’s resonating — or isn’t.Think of it as the differencebetween watching lazy, well-fed lionsat the zooversus being on safari and observing themhuntingin the wild.
Only the safari delivers the big picture necessary touncover thefrequency andseverity of usability flaws— while also identifying potentialopportunities to achieve higher levels of customer satisfaction.
Think Safari Not Zoo
Here’s a real-world example: A high-end consumer electronics manufacturer relied on a visually rich product page to telegraphitsproduct benefits quicklyto customers. To ensure that the advantages were as clear as possible,the companyconducted traditionalusability testing, as well as page evaluations usingadvanced user experience technology.
During usability testing, whensubjects were asked to perform a specific task on the product page they did so witha minimal amount of scrolling and zeroed in on the page’s video elements.These findingsmade senseto the company, given the good conversion rates it was already seeing from its videos.
Good Enough Isn’t Good Enough
However, when a far more comprehensive population of visitors wasmonitored using advancedCXM techniques, a dramatically different conclusion emerged. A significant number of visitors were very interested in content that was located toward the bottom of the product page —so many in fact, that the bottom of the page was drawingmorecustomers than the videos.
What’s more, aggregate data represented in heatmaps and confirmed by drill-downs into individual anonymous session replays revealed that many visitors skipped over the middle of the page to find content that appeared only at the end.
The implications for page performance were clear: When the site removedits poorly performingmid-page content andmoved interestingcontentupfrom the bottomto feature it more prominently beside thewell-performing videos, overallpage performance improved dramatically.
Learning Opportunities
Webinar
Jun
6
The Future of Social Media: Emerging Trends and How to Stay Ahead of the Curve
The impact of emerging technologies like AI on social media
Webinar
Jun
7
The Building Blocks of Personalization
This webinar will explore how personalization can create powerful ecommerce shopping experience.
Webinar
Jun
8
Attracting—and Keeping—Your High-Net-Worth Investors
Join PwC to dissect their recent survey findings on a specific customer subset — high-net-worth investors.
Webinar
Jun
13
The Future of Contact Centers with AI at the Helm
Learn how to balance human intuition and interaction with the potential uses of AI to create delightful and lasting CX
Webinar
Jun
20
3 Steps to Unlock Your Customer Data and Drive Revenue
Discover how Twilio Segment’s #1 Customer Data Platform helps implement data-driven strategies.
Webinar
On demand
AI for Marketers: Optimize Personalization Across the Funnel
Understand the benefits of utilizing AI across your entire marketing stack
Subjectivity Injects Bias
Usability testingis inherently subjective, as theoverly detailed feedback I gave during my college experiment richly illustrates.Much like my younger self, people in usability testing situations are often overly focused on pleasing the observer — who may be paying them — by generating lots of detailed feedback.
In short, usability testers tend to be more stringent, observant and critical than they would actually be when shopping from their own computers or smartphones. That’s whyit’s important to keep thedifference between what people say affects them and what really affects them firmly in mindat all times .
Objectivity, Context and Scalability
As our case study shows, observing usersanonymouslycan provide a far more complete and fair picture of actual usage scenarios. When users don’t know they’re being evaluated, there is no chance of “contaminating” results with user presuppositions,something that traditional usability testing simply cannot deliver.
What’s more,on large-scale online retail sites, looking at user behavior on a single page ignores the broader context of the online experience —the overall customer journey. To truly understand whyusers behave in certain ways,w e need to take into account how theyarrived atapage and where they are going next.
Finally, and perhaps most important, the sheer scale of digital commerce brings the cost-effectiveness of standalone usability testing into question. Conducting usability testing for 10 subjects three times a year carries a significant price tag. This same sum could be used to monitor literally a thousand times that number of users on a daily basis todrive the agile interaction design that is crucial to keeping conversions high.
Let Your Objective Determine Your Approach
Thebottom line is to understand your marketing objective before determining which approach makes sense.Let’s be clear: Usability testing plays a key role in measuring high-level, global and impression-based issues that may impact a brand, such as new logos or new product packaging.
Small-scale, face-to-face usability sessions can pick up on visceral reactions and observable responses that reflect the actual emotional reactionsof the testing subjects, somethingthat technology cannot yet do on an individual scale.
CXM Generates Higher ROI
However, on the global stage facing millions of digitalconsumers —and with billions of dollars in sales and shareholder value at stake —it’s important to know when a hybrid approach like digital customer experience management is the better choice.
Chances arethe ROI of optimizing according to the aggregate data of millions of users will be far greater than the limited-scale improvements that can be achieved via a usability lab.
Learn how you can join our contributor community.
