science experiment
High-performing marketing programs are built on a foundation of experimentation. PHOTO: jdn2001cn0

For decades, brands have relied on gut intuition and over-generalized best practices to drive their marketing programs.

But sustainable success in today's competitive landscape requires something more sophisticated than a strategy driven by the highest paid person's opinion. High-performing marketing programs are built on a foundation of experimentation — testing and optimization are threads that run through every campaign.

It's fair to say that most brands are familiar with the concept of testing.

Experimentation: A Key Tool

In a 2016 Monetate survey, 59 percent of marketers interviewed said that they are already using A/B or multivariate testing as a strategy to personalize digital experiences for consumers. Indeed, experimentation is an integral part of any effective marketer's toolkit.

So if adoption and enthusiasm are no longer major hurdles when it comes to testing, what is keeping brands from maximizing the value of their investment in experimentation?

Consider this client, a producer of consumer technology products who used experimentation to test the effectiveness of different variations of CTA messaging on their e-commerce site. Results showed that in terms of click-through rate, one variant outperformed the rest by 30 percent and further, this variant generated a 200 percent lift in revenue for the brand.

Well, you’ve read case studies like this with one raised eyebrow. Your skepticism is likely driven by the understanding the devil is in the details.

Often times the content does little to elucidate any of those details that would help a marketer understand how they too can see these exceptional outcomes in their experimentation programs.

You are left to believe that "if you buy the tools, the results will come."

Design of the Test Is Critical

What ultimately determines the usefulness of a test is its design.

A well-designed test is based on a solid hypothesis; is performed in a controlled environment, and has well-defined parameters.

Marketers have to see the investment in test design as a non-negotiable part of an effective program. This may mean calling in experts who understand the statistical nuts and bolts needed to come up with objectively good experiment ideas.

A sound investment would not just be focused on tools but would also include consultation with subject matter experts to guide a brand through this process.

Benefits of Experimentation

A commitment to a solid experimentation program saves businesses from the catastrophic fallout of making bad decisions based on unreliable test results.

Moreover, driving a marketing program with the dubious results of a poorly designed experiment undermines the very premise of using testing in the first place. Even if your poorly designed test happens to drive a positive lift in performance, those results certainly won’t be repeatable.

When businesses buy into these foundational ideas about experimentation – an ethos begins to develop.

That mindset infiltrates marketers and other decision makers — you begin to see proof in how controlled experiments can positively impact their bottom line. And as the enthusiasm for testing spreads throughout their organization, so does the demand for executing more tests.

Eventually, there will be more test ideas than what can be reasonably executed in a period of time.

Brands need a more sophisticated way to prioritize experiments than first come, first serve. An optimization framework takes the guesswork out of prioritizing ideas and is far more reliable than going with just an opinion or a whim about what to do next.

Establish the Right Criteria

To prioritize effectively, brands can establish a set of categorical and quantitative criteria that you will use as a mechanism to evaluate the priority of a given test idea. For example, you may have two test ideas related to the effectiveness of variations of an email subject line.

A way to prioritize one idea over another would be to understand the relative sizes of the target audiences for each test. In this scenario, with all other factors aside, the test idea that targets the largest audience would likely be the one to prioritize.

For the same consumer technology client, throughout the life of a -two-year experimentation program, a litany of marketing insights was uncovered. Here are some examples:

  • For video content, a thumbnail preview coupled with a large play button is the best format for driving engagement
  • Mid-page email sign-up blocks are more effective at driving leads than an expandable form of interaction
  • CTA text engagement differs by audience segment (geo, journey phase, etc.)
  • Discount messaging expressed as $ OFF is more effective than % OFF

When the aforementioned elements come together to establish an experimentation program, longer term benefits begin to reveal themselves.

As tests are completed, the results become an engine for marketing insights.

These insights can be collected into a rich knowledgebase that decision makers consult to inform strategic planning for media spend, site, or app design, as well as audience targeting.

With so many tools, technologies and normative statements dictating what brands should or shouldn’t do to drive strong performance in their marketing programs, it's hard to find objectively useful guidance.

Marketers who are successful in helping their companies maintain a competitive edge know that tools are not enough. You must create a culture of experimentation where the best ideas always win if they feed into well-designed experiments.