According to the a new Forrester Wave report, online testing technology is undergoing a rapid evolution where it is becoming a core competency of digital marketers, while leading online testing platform providers are starting to follow the lead of digital marketing vendors by offering functionality that support multiple channels and create consumer profiles.
Online testing is turning into a core digital marketing competency because of the valuable support it can offer optimization efforts. Advanced online testing solutions evaluate customer experiences against a control group using statistical data to help marketers determine how to provide an optimal customer experience.
In the report, Forrester Wave Online Testing Platforms Q1 13, Forrester identifies several “next generation” capabilities provided by mature online testing technology providers. These include distributed access to support a wide range of users who may not be technical experts (such as marketers) as well as to support the rigors of online testing activities, sophisticated multichannel testing capabilities, the ability to collect, store and distribute visitor data at a granular level, advanced testing scenarios that encompass multiple pages, domains and sophisticated success metrics, automated core testing processes and analytics and integrated suites of optimization tools.
Forrester identifies seven online testing vendors as standouts in the field. In order of rank, they are Maxymiser, Adobe, Autonomy, SiteSpect, Webtrends, Monetate and Optimizely. Following are summaries of Forrester’s review of each vendor, divided into the “Leader” and “Strong Performer” categories.
The Leaders - Maxymiser, Adobe, Autonomy, SiteSpect
Maxymiser receives top honors among all online testing vendors from Forrester due to its “consistently strong capabilities across all categories of online testing functionality.” In particular, Forrester singles out Maxymiser’s reporting and analysis functionality, as well as mobile and social testing. Furthermore, Maxymiser receives credit for focusing on the US market to the point of relocating executive management to New York despite being a UK-based company.
Forrester says users who need a balance of professional services and advanced technology will most benefit from Maxymiser, but cautions the vendor must “execute on its vision for comprehensive optimization and visitor data management” to maintain its category lead.
The Test & Target component of the Adobe Marketing Cloud suite is the runner-up among online testing solutions reviewed by Forrester, who calls it a “solid product across all facets of online testing capability.” Test & Target also benefits from Adobe’s strong ecosystem of third-party partners and existing integration with commonly used Adobe analytics and content management technology. Forrester says current Adobe users have most reason to use the application and advises Adobe to improve ease of implementation and speed up its product development timetable.
The Optimost component of the Autonomy Promote suite comes in third. Running on the Autonomy Intelligent Data Operating Layer (IDOL) analytics platform, Optimost gets kudos for “strong testing capabilities featuring a variety of optimized techniques.” Forrester expects users with highly advanced testing requirements to get the most value from Autonomy Optimost and suggests that Autonomy update a “badly dated” user interface and “address ongoing market uncertainty” as a result of its 2011 acquisition by HP.
Although SiteSpect gets attention for its tagless deployment model, which Forrester deems a “polarizing” market differentiator, the report praises its “very strong testing capabilities” and especially recommends the solution to technical users. SiteSpect needs to improve usability and capitalize on the advantage tagless deployment offers in testing emerging channels such as mobile.