Digital marketers seeking new ways to gain a competitive edge should look at capital markets firms for inspiration. Financial traders’ rapid embrace of real time analytics and data visualization holds many lessons for tackling the data challenges that now hamstring the effectiveness of online marketing.
The Vanishing Luxury of Time
Capital markets experienced a technological transformation towards real time analytics following the last market crash. Pre-crash, there wasn’t an urgent need for real time data analysis for things like understanding risk exposure and portfolio structures because latency was built into the system. Traders had the luxury of time. They could wait days to read reports and other market data before evaluating and modifying their positions.
But then the market bubble burst and those firms that were heavily invested in subprime derivatives were catastrophically, if not mortally, wounded. Those who survived the crash knew they had to take latency out of the equation. They had to quickly adopt new technologies that would let them evaluate their portfolio and risk profiles literally as they were changing.
A similar phenomenon is now overtaking marketing. Before the Internet and the ubiquity of mobile connectivity, the industry was heavily weighted on print; latency was built into the marketing plan. Companies could kick back and observe trends over a month’s or quarter’s time and adjust their marketing programs accordingly to address the results.
With the emerging dominance of online marketing -- especially where key younger demographics are concerned -- latency is becoming a debilitating condition. There can no longer be any delays between the time a potential customer decides to purchase or engage with a company and the time a marketer sees it happening. If marketers cannot capture that consumer’s mindshare the moment it is forming, they face the same fate as pre-crash capital markets.
The Big Data Challenge
But as financial traders learned, real-time analysis is a complicated big data challenge. The ability to analyze consumer activity in aggregate across an entire market involves high volumes and varieties of data points. Even more challenging is the velocity of that data. Marketers need new tools for making sense of data “in motion” in real time.
Capital market firms learned that traditional data warehouses and business intelligence systems don’t cut it because they require capturing, transforming and loading the data into a database that can then be queried after the fact. In the rough and tumble marketplace of online retailing, even delays of a few seconds puts marketers at a competitive disadvantage.
Marketers need up-to-the-second visibility into who their website customers are, what call-to-actions or content they engage with, and what ads and incentives will trigger them to convert. They also need to assess how their affiliate networks are driving traffic, as well as pulling in more qualitative unstructured data from social media, mobile applications, Amazon reviews and the like. Wait too long to sort out all these data points and competitors will step in and steal customers away.
For financial traders, getting latency out of the decision chain was the primary impetus to radically transform their technology platforms. The typical transform and load latency of database-centric systems were supplanted by new kinds of real time analytic tools that could capture and visualize huge volumes of raw data streams in motion; everything from complex event processing (CEP) engines and message queues, to tick databases and unstructured social data.
Traders could now evaluate large portfolios of complex financial instruments in real time for risk exposure, liquidity management, rogue trading and other factors with split-second accuracy because data visualizations provided the visual filters for revealing unseen patterns and outliers.
Latency in Marketing Automation
After purging latency problems, however, traders faced another upheaval in their world; a change that is only now coming to full force in the world of marketing -- automation. While the image of roiling traders shouting orders on a frenzied trading floor is practically a cliché, the truth is that most trading today is carried out by jousting computers executing sophisticated algorithms. A similar phenomenon is overtaking online marketing today and CMOs would do well to take a page from the financial world’s playbook on this front as well.
For marketers, latency in an age of programmatic advertising is still the enemy -- perhaps even more so. Take real time bidding, for instance. Algorithms rule when offers and opportunities to buy ad placements are being made on the fly in the time it takes a prospect’s Web page to load; human timeframes for decision making are simply out of the question.
Therefore, just as in the world of financial services, urgency moves to algorithm management; understanding what’s working, what isn’t and why. And, as traders learned, that insight must be delivered in a timeframe that matters, but also with context. So, while speed-to-decision is crucial, simply seeing what’s happening in real time doesn’t always provide enough information. To do the job today, marketers must be able to see and judge in real time what’s happening right now in the context of what has happened before.
In the same way that network operations centers instrument components for anomalous behavior, financial traders create algorithm thresholds to set alarms when outcome goals are being missed. Marketers must be prepared to engage the same techniques; visually drilling into data underlying the alarm to understand what’s not working, bringing in historical information to provide context and then using that insight to adjust a programmatic buy on the fly. Waiting until the holiday shopping season is half over is no longer an option.
The divergence between the automation transformation in financial services and the one rapidly overtaking marketing comes in the algorithm hashing, or who manages the tweaking of the programs and how. In the financial services world, it’s no secret that the math wonks are taking over. Gut feel and instinct is being replaced by game theoreticians and formulas derived from advanced physics. In marketing, the explosion of available data is similar, but the ultimate target is still fickle human tastes and behaviors which is likely to limit the influx of pure math geeks into marketing. For now.
As customer touch points expand making more inventory and information available and audience fragmentation increases, the marketer’s calculations will only become more complex. The rise of the Internet of Things will, as a small example, allow automobile navigation systems to communicate with electronic billboards in real time.
Engaging with customers across these ever expanding touch points will require an increasing reliance on an automated, algorithmic approach to buying -- and a context-driven, real time view to manage and drive it all.
Title image by Stuart Monk (Shutterstock)