Feature
In marketing circles, the classic A/B test has a long and honorablehistory. It’s been around since the 1930s, when mail-order advertiserswould routinely run tests in magazines to see which version of an adpulled the most coupons. Back then, of course, testing the efficiency ofone element at a time was sufficient. Now, online marketers need tomeasure multiple variables -- all at once and in relation to one another-- to coax more sales and conversions from their websites. This can’tbe achieved by A/B testing. Hence the emergence of multivariate (MVT)testing.
So which one is right for you? Well, it depends on what you’re testing and what you need to know. Consider the following as you make your decision.
If you simply need to know which headline on your landing page will generate more click-throughs, downloads or sign-ups, A/B testing might be the right choice for you. But if you need to know that, as well as which price presentation delivers higher sales, and with what headline the winning price presentation performs better, you’re going to have a lot more success (and save a lot of time) using multivariate testing.
A/B testing is simple and easy to understand and the tests are relatively easy to conduct in-house (if you prefer). And it’s still better than guessing or going on instinct.
Marketers who live and die by their online results, however, need a more complex solution and therefore generally rely on multivariate testing. Multivariate testing applies to a multitude of page combinations, opening up many more possibilities for understanding how your site works for visitors, and how to continually refine it.
Here’s where multivariate testing pays its way.
MVT Reveals How Your Site Elements Interact
Unlike A/B testing, multivariate testing is particularly good for understanding the interplay between various elements of your customer experience. For example, let’s say you do an A/B test on two or three headlines for a landing page. You then take the “winner” and continue testing with two or three calls-to-action (CTA), perhaps, or different product images.
The trouble is -- as marketers discover all the time -- it’s entirely possible that the optimum solution is in fact your original headline but with a particular CTA and graphic combination. A/B testing can miss that entirely. Or take forever to figure out.
The multivariate approach lets you explore various combinations of your page elements at the same time -- headline, graphic, CTA, product ratings, navigation -- to ultimately arrive at the most productive solution. As often happens, an optimum combination of changes typically lifts conversions more than a single change of headline or visual.
MVT is Better for Exploring
Let’s say you’re faced with a homepage or a checkout sequence that doesn't seem to be performing well. What should you start working on? Where are the soft spots? A well-designed multivariate test allows you to experiment with dozens of elements to see which of them move the needle the most -- and in what combination. It’s not uncommon for marketers to run tests involving as many as 96 variants, trying to understand the dynamics of a page or sequence.
Learning Opportunities
Webinar
Jun
6
The Future of Social Media: Emerging Trends and How to Stay Ahead of the Curve
The impact of emerging technologies like AI on social media
Webinar
Jun
7
The Building Blocks of Personalization
This webinar will explore how personalization can create powerful ecommerce shopping experience.
Webinar
Jun
8
Attracting—and Keeping—Your High-Net-Worth Investors
Join PwC to dissect their recent survey findings on a specific customer subset — high-net-worth investors.
Webinar
Jun
13
The Future of Contact Centers with AI at the Helm
Learn how to balance human intuition and interaction with the potential uses of AI to create delightful and lasting CX
Webinar
Jun
20
3 Steps to Unlock Your Customer Data and Drive Revenue
Discover how Twilio Segment’s #1 Customer Data Platform helps implement data-driven strategies.
Webinar
On demand
AI for Marketers: Optimize Personalization Across the Funnel
Understand the benefits of utilizing AI across your entire marketing stack
While that may sound dizzying, a multivariate test will rather quickly point to elements -- or combinations -- that have the greatest effect. (Or just as valuable, you’ll see which elements don’t affect much at all.) The results and patterns will lead you directly to a better-performing page, as well as inform your design and development decisions going forward. You understand more than you ever could with A/B tests.
In a recent test, a rental company discovered it could increase sales of “extras” by 25% by adjusting the size, positioning and volume of product information -- revealed by multivariate testing.
MVT Fosters a Test-and-Learn Culture
What would improve your multi-page checkout sequence? Do returning visitors respond differently than first-time visitors? Should your landing pages differ for customers coming from different sources? What should you change? Should you ask for more information on membership sign-ups? Or less?
The most successful digital marketers ask questions like these all the time -- and continually test, refine and optimize what they’re doing. It’s not about wholesale makeovers but continuous iteration, driven by constant experimentation. Using multivariate testing not only provides a platform for rapid exploration and validation but also creates an environment for creative teams, marketing and development to keep asking, “How can we make this better? What can we try, right now?”
That’s how the digital elite think.
Title Image courtesy of Lightspring (Shutterstock)
Editor's Note: To read more of Mark's tips for fine-tuning your website, see his Let Your Customers Optimize Your Website for You
