Look Before You Leap Into the Big Data Revolution

Like the proverbial elephant and five blind men, big data looks different depending on who's describing it: from holy grail for information in society, to dangerous step toward a world in which people are treated based on predictions of what they are likely to do instead of what they actually do. While neither of these extremes may turn out to be our future, big data is obviously more than a passing fad, and will be defined ultimately not by what it is but by what is done with it and what it does to those at whom it is directed.

From what we know now, we can assume that big data’s impact on us will be roughly proportional to the extent to which it ushers in an unprecedented level of analysis in the culture. The term analytics is being increasingly linked to big data on the implicit assumption that their intersection will enable predictions about human and organizational behavior far beyond what is or has ever been possible. A strong case can also be made that without analytics and its results, big data is just … well, lots of data. So we’ll use the term big data analytics in this article.

Why is there such interest in big data analytics? Does it do all that it claims it can, and if so, are the results any better than what's currently possible? The interest seems partially driven by what it will mean to segments of industry most likely to benefit from its growth. For instance: to computer manufacturers, big data analytics could mean demand for big, expensive servers and storage arrays; to the communications/cloud industry, the movement of much more data and many more dollars; for the IT services industry, more contracts for more money developing and applying analytical techniques, perhaps extending even to collecting and storing the data on a service basis; and to the content management software industry, big data analytics could mean expensive additions to its existing products. For these sectors, big data analytics could be the gift that keeps giving.

Big Data Analytics: Real Changes and Real Impacts?

What will big data analytics mean to the rest of us us? What's noteworthy about big data analytics is less about its touted benefits -- or dangers -- and more about some of its less discussed impacts.

Perhaps most important, big data analytics, if it catches on, will fundamentally reverse the roles of buyer and seller, taking us from a culture where people buy based on what they know or believe about prospective sellers to one in which sellers influence buying activity by what they learn about their prospective customers.

The impacts of that kind of change could be far-reaching and unpredictable.

For starters, big data analytics-based marketing could accelerate the move from advertising and PR based on broad demographics and mass media to individual communication with individual prospects based on sellers’ analytical predictions about what will be most effective. While this kind of targeting is practiced today at a fairly gross level, taking it to a focus on individual prospects with individual messages is light years beyond current practices.

The eight and nine-figure budgets now spent on advertising via electronic, print and display media could become eight and nine-figure costs for data collection and analysis with impacts more dramatic than you might think. What if all those TV programs and made-for-TV movies you love could no longer find sufficient sponsor dollars to cover their production costs? Virtually all programming could become subscription based and available only to those viewers who coughed up a monthly fee -- see TV in the UK for an example.

In the age of analytical marketing, we could see a rush by firms to collect more and more data in an effort to more finely tune their analysis and increase their competitive edge. This push for more data could become increasingly intrusive -- some of it actually crossing the line of legality -- possibly even giving rise to a shadowy industry “laundering” and selling not-quite-legally obtained data. And as more and more data is collected and aggregated, the market for cybercrime would expand and the hackers would become more capable as the rewards for a successful intrusion grow.

A Predictive Society?

The other side of the coin: industry and government treating individuals based on predictions about their future behavior. Think paying more for health insurance if you smoke is inconvenient? What if health insurance companies began looking at your activities across the board and deciding you are not a good risk based on some third or fourth level analysis of your habits.

How about being refused auto insurance -- or a drivers license -- based on a pattern of texting while your phone’s GPS says you may be in a vehicle, or being refused a gun purchase because ATF’s analysis noticed that you attended a Tea Party rally last year? Governments have shown a sometimes eerie willingness to treat citizens based on assumptions rather than reality. Being charged with a “hate crime” not based on the crime itself but on an assumption of what the perpetrator was thinking at the time could be only the start.

And a Disgruntled Population?

And there is the push-back as people realize that their every move is being trapped and crunched to predict their next move. Even people who behave like lab rats don’t often respond well to the realization that they are being treated like lab rats. History shows this kind of backlash usually goes to extremes before cooler heads prevail. We're seeing the rise of smart phone apps designed to prevent unauthorized access to calls and texts, and with the recent invasion of widely used apps like Snapchat coupled with the huge hacks of retailers like Kmart, Home Depot and Target, the public is becoming more sensitive to the fear that what happens in their personal cyber world doesn't necessarily stay there.

If this trend continues, we risk widespread rejection of many of the tools and gadgets that have come to define how we communicate and buy. The adoption of a big data analytics approach to dealing with customers would be like rubbing salt in an already open wound.

So What Should We Do … or Not?

We are on the cusp of a cultural change that goes far beyond what the big data analytics promoters are discussing. If futurists like Ray Kurzweil are even close to being correct in their straight-faced -- and I believe overly optimistic -- projection that computer power will eclipse human intelligence by the 2040s, the level of analysis available from big data analytics will become irresistible, whether accurate or not.

Society should recognize big data analytics for what it is: a fundamental cultural shift that will affect us in ways we can’t fully predict or understand but must be ready for.

We aren’t hearing things like that from big data analytics’ supporters, perhaps because they stand to gain so much from its adoption. What we hear from the doomsayers seems so fantastic that no one much listens. There is a middle ground on which we can, and I think must, meet to explore what the intersection of big data, analytics and exponentially growing computer power (see Moore’s Law) will mean for our lives and our future. Only with that kind of clear-eyed examination can we prepare ourselves for the kinds of changes that could make the rise of social media seem positively trivial.

As for firms considering the prospect of adopting big data analytics for their marketing efforts, “cautious advance” might be the appropriate watchword. There is probably a level of enhanced data analysis that will improve sales and profits, but it may not require jumping into the deep end of the big data analytics pool just yet. Amazon, for example, does a good job of showing you things you might be interested in when you make a purchase, but that probably didn’t require full-blown big data analytics to conceive and implement. William of Occam’s aphorism -- paraphrased here -- may still be words to live by: “all things being equal, take the simplest solution.”

Take note of what is and will soon be available, then figure out how you can make things work better -- without finding yourself out on the windswept plain of “too early adopters.”

Title image by John P (Flickr) via a CC BY-NC-SA 2.0 license