Editorial
Let’s face it. Everyone loves to be the smart one in a given instance.
But A/B testing can sometimes leave people scratching their heads, feeling anything but smart — especially about planning resources efficiently.
So what can you do? Start with your homework. Understand what is being tested and account for how the teams are potentially impacted.
A/B Testing Basics
A/B or split testing involves comparing two versions of a web page to see which one performs better.
The difference between the pages is a web element that is of interest to a marketer — anything from the shape of a button to text or an image.
- A is usually a control for the test, the original layout or media element
- B is the layout that contains the changed element
The test works by showing the two variants to similar visitors at the same time. The one that gives a better conversion rate is the preferred choice.
Getting Started With A/B Testing
Marketers should consider the following ideas before launching a test session.
Make Sure There Is Enough Traffic to Achieve the Required Sample Size
An A/B test requires a certain level of site traffic or app use to reach a needed sample size for a test. But some organizations launch tests without checking whether the traffic levels will provide significant results within the testing time.
Determine a sample size estimate for your test. There are a number of ways to calculate this scientifically. Online sample size calculators, such as those from Wingify and Optimizely, simplify the process.
The estimate can then be compared to the current website traffic or app usage. If the difference between the estimate and the media is large, more activity — more marketing for a site or enticing customers to use an app — is required to achieve the target sample size.
Moreover, you can use real-time reports in an analytics solution to confirm how frequent traffic is contributing to your A/B test pages. The activity can confirm the duration planned for a test and alert if the decreases in activity will hinder attracting the target observations.
Learning Opportunities
Webinar
Jun
13
The Future of Contact Centers with AI at the Helm
Learn how to balance human intuition and interaction with the potential uses of AI to create delightful and lasting CX
Webinar
Jun
13
Learn How to Deliver Dynamic Video Experiences at Scale
Learn how to convert existing assets into video experiences in near real-time.
Webinar
Jun
20
3 Steps to Unlock Your Customer Data and Drive Revenue
Discover how Twilio Segment’s #1 Customer Data Platform helps implement data-driven strategies.
Webinar
Jun
21
Retail Reinvented: A Masterclass in Creating Digital Experiences That Click
Hear how online retailers architect their digital experience infrastructure for peak traffic and demand
Webinar
Jun
21
The Feedback Loop: Accelerating Personalization for Exemplary Customer Experiences
Are you ready to harness the power of feedback in your personalization strategy?
Webinar
Jun
22
How to Succeed in Today's Engagement Economy
Doing true personalization at scale with marketing automation
Webinar
Jun
13
The Future of Contact Centers with AI at the Helm
Learn how to balance human intuition and interaction with the potential uses of AI to create delightful and lasting CX
Webinar
Jun
13
Learn How to Deliver Dynamic Video Experiences at Scale
Learn how to convert existing assets into video experiences in near real-time.
You can also use analytics reports to monitor if your audience is from the right persona — demographic, location and interest can help ensure that you are not reaching an alternative population to who the changes were meant to benefit.
Group Media Elements Into Ideas You Want to Test
Testing every single element undermines what an A/B test is meant to accomplish. Instead, test one idea against the current production design, with each idea featuring a group of element changes meant to address a specific design concern.
For example, a test can be of an app or mobile page with combined element changes for the most visible section of a screen, as opposed to just changing a button.
Review What Influences Teams (Make Sure the Stock Room Is Ready, Too)
At first blush an A/B test does not sound like a high effort activity, but like any digital changes, it can trigger a lot of technical implementation “behind the scenes” of a live site or app.
With each task, marketers should map out roles and ask, “Who else is impacted with this test?”
If you are testing pages that involve products on an e-commerce site, make sure that your backend operations to ship that product are ready. The last thing you want to do is run an A/B test while creating new headaches in order fulfillment.
Overall, you should understand what has to happen to manage the tasks well.
Testing has become an increasing part in verifying if a webpage or app change will improve a customer experience. With sales at stake, organizations should leverage testing as a way to bring teams together to improve the delivery of those improvements, too.
Learn how you can join our contributor community.
