Feature
Once you become immersed in testing your website it’s not hard to get caught in a matrix of never ending variables and what if scenarios. While there are many advanced tools that can help companies evaluate the usability of a website -- a few of which we’ve already covered -- ultimately there are two predominant types of testing methods that can be used: Multivariate and A/B Split Testing.
Today we examine these two testing concepts.
Defining Your Site’s Goals
Before we can effectively investigate these methods, let’s first establish what our goal is. Typically, a company wants to convert users into customers, which of course means revenue. Unless a user can actually click and purchase a product directly from the website, the process of converting user to customer can take an indirect route. Maybe there are forms a user can submit that signal interest or perhaps it’s more about building customer and brand loyalty by getting users to share information via social media. Take a moment to spell out the process by which your company hopes to convert users into customers.
For example:
Home Page → Product X → Sign up for Online Demo → Thank you page
Home Page → Product X → Purchase Product → Registration form → Thank you Page
Choosing Your Testing Tool
A/B Split Testing
A/B testing (also referred to as bucket testing) is the process of testing different versions of a specific page with only one element different between the pages.In other words, it tests the effectiveness of one landing page over another. Normally the current landing page will be used as the control, and a second page with some changes to the original will be used as the experiment.
What used to be a rather tried and true tactic of the direct mail industry, has permeated its way into website usability testing.Often companies will employ this method by offering two or more different user interfaces randomly to their audience. Evaluating what interface drives users through a site’s desired path can help guide website navigation and IA decisions. Retailers like Amazon and publishers like the BBC have been known to use this method.
The process for implementing and analyzing A/B Split Testing is fairly straightforward.
Step 1: Identify the Page to Test
When choosing a page to test, it’s recommended that you choose a page with high traffic so you can gather data faster and make a conclusion. Once chosen, decide on one element that you would like to use as your testing element.
Step 2: Choose the Conversion Page
Decide on a desired goal that you want to track, whether it’s a contact form submission, a download, a purchase, a sign-up or a time-on-site goal, etc. When tracking a form submission, purchase or a sign-up, you will want to have a unique “thank you” URL that you can use as your completion page. This is the URL you will add in the goals.
Step 3: Set Up Tracking Scripts
In order to track your experiment properly you will need to add tracking scripts to your control, test, and goal pages. Two types of scripts are needed: control and tracking. The control script makes sure that the experiment variations are switched randomly and that all variations are displayed an equal number of times, while the tracking script ensures that visits to both the test page and the conversion page are tracked. Implementing tracking code will vary between software.
Step 4: Decide on A/B Distribution
Depending on the number of tests you are running, you need to decide on what percentage of your traffic will be displayed the control page and version A, version B, etc. If you’re testing 2 pages, splitting the traffic up with 50% for each page is the easiest way. It is recommended that you use only one variable (i.e. your normal page, plus a page with one thing changed).
Step 5: Analyze Results
Without analyzing the results of your test, your efforts don’t mean much. Devise a plan to implement changes to your website based upon your evaluation. Many software testing solutions provide reporting features that allow you to see which variation was more successful.
Multivariate Testing
Multivariate testing is the process of testing more than one component of a website at the same time in a live environment. In essence, multivariate testing lets you take conduct several A/B tests on one page at the same time.
This graphic shows how each user is shown different elements on a page in multivariate testing. (Courtesy of ProBlogDesign.com)
While the only limitations for multivariate testing are the amount of time it will take to gain sufficient data to come to a reasonable conclusion, let it be known that the more components you add to a test, the longer and more data you will need to complete a test.
Typically employed in an effort to ascertain which content or creative variation produces the best improvement in the defined goals of a website, multivariate testing can show startling behavior changes through testing different copy text, form layouts, landing page images and background colors.
Most multivariate testing solutions are based upon Javascript technology, with Google Website Optimizer, being among the most widely used multivariate testing platform. To implement, each area of your web page that you want to test is defined with Javascript with alternatives to each location so that goal pages can accurately track which version performed best. When users visit your site, they get served different versions. The version that they received is populated within their cookies, this way on subsequent visits they receive the same version.
Learning Opportunities
Webinar
Jun
6
The Future of Social Media: Emerging Trends and How to Stay Ahead of the Curve
The impact of emerging technologies like AI on social media
Webinar
Jun
7
The Building Blocks of Personalization
This webinar will explore how personalization can create powerful ecommerce shopping experience.
Webinar
Jun
8
Attracting—and Keeping—Your High-Net-Worth Investors
Join PwC to dissect their recent survey findings on a specific customer subset — high-net-worth investors.
Webinar
Jun
13
The Future of Contact Centers with AI at the Helm
Learn how to balance human intuition and interaction with the potential uses of AI to create delightful and lasting CX
Webinar
Jun
20
3 Steps to Unlock Your Customer Data and Drive Revenue
Discover how Twilio Segment’s #1 Customer Data Platform helps implement data-driven strategies.
Webinar
On demand
AI for Marketers: Optimize Personalization Across the Funnel
Understand the benefits of utilizing AI across your entire marketing stack
Selecting One Approach Over Another
While both useful, each of these tests present unique challenges and are not always appropriate for all kinds of testing. How do you know when to use either of these methods?
If you’re site is small, both in terms of pages and visitors, A/B Split Testing will work best because it is more manageable to evaluate and interpret changes in behaviors. As well, if you want to test basic changes on the site, such as a video on a landing page versus a photo on a landing page, A/B Split Testing will be able to show you results that are easy to understand and analyze. However, because testing is rather basic, users may not be able to understand the reason why certain behaviors occurred.
Multivariate testing is best used for bigger sites with large amounts of traffic. While large companies can definitely employ A/B Split Testing effectively, usually there is more reason to test multiple variables on one page.While multivariate testing will help you best to understand what affected user behaviors on a site, it is a very involved process that takes time to implement and analyze.
Identifying the Right Software
Now that you know what kinds of tests to run on your websites, next comes the how. There are many choices, of course. As well many web content management platforms already include testing options. Here’s a brief summary on what some vendors provide.
Integrated With WCM
Sitcore
Sitecore’s Online Marketing Suite provides visitor experience analytics, which offers content profiling feedback, allowing users the ability to adjust content profiles rapidly, evolving as you learn about your visitors and their behavior, including the sources they came from, what they searched for on and off site, the goals they reached and content they consumed. Sitecore’s Real-Time Site Personalization makes classic multivariate and A/B split testing configurable out of the box, and provides content profiling dialogs that allow you to capture rich insights on what your customers are interested in, and who they are.
Autonomy Interwoven
Autonomy Interwoven's TeamSite combines WCM and Multivariate Testing in a single interface and allows marketers to quickly make adjustments that lead to proven increases in online conversions. As well, Interwoven provides businesses withinsight on all forms of information, and the capabilities to act on new opportunities as they emerge, and continually optimize these interactions to maximize results.
Day Software
Day’s CQ 5.3 is a content management solution that enables campaign targeting and optimization. Users can create, manage and schedule campaign teasers targeted to different customer segments to drive click-throughs to campaign microsites and landing pages. CQ 5.3 also provides integrated support for A/B and MV testing to enable Marketing to observe preferences and optimize campaign promotions for higher conversion rates.
Third Party Solutions
There are also a plethora of third-party tools that companies can use to conduct their testing. Here are a few to check out:
Omniture
An Adobe company, Omniture lets users optimize their site through various tools like Test&Target, which lets marketers quickly and continually execute multiple A/B and multivariate tests, measure effectiveness and relevance of content across any online channel and increase content relevance through segmentation, targeting and automated personalization.
Webtrends
Webtrends’ Optimize provides comprehensive testing, targeting and site optimization in a single solution. Users can choose from A/B/n or simple multivariate tests and graduate to campaigns that use sophisticated targeting and personalization, designed test and optimize web content, such as landing pages, funnels and registration pages.
Google Website Optimizer
Google’s Website Optimizer is a free website testing and optimization tool, which allows users to test and optimize site content and design. By identifying your most effective site elements users can test everything from individual copy blocks and images to complete page layouts. It features a graphical reporting interface designed to show how each aspect of your test is impacting your site’s goals.
Bottom Line
The more you test, the better the opportunity your company has at improving its site and the user experience. Making sure that you’re appropriately implementing tests designed for your website can ensure that the results impact your site’s users effectively so as to convert them successfully into customers.
Are you currently using a specific testing software? Tell us about it.
