Big Data as a Disrupter Thinking About Big Data Strategically

To do that, we need to put on our management consulting hats and enlist the help of a simple but powerful two-by-two model. The dimensions of this model result from two basic questions relating to the big data itself and the algorithms that are applied to derive actionable insights from the data:

  • is the data newly available or is it existing data, and
  • are the algorithms applied to the data newly invented or are they existing?

Those two simple questions form the basis for our big data-as-a-disrupter model:


Big Data Success Stories

To get a sense of how the model can be applied, let’s first touch on a few of the major success big data stories of the past, before the term big data was even cool, and examine how they map to our model. Success story number one is Google -- arguably the most successful IT company of the last decade.


Google had the insight to use otherwise unexploited data (the links people built between web pages) to infer web page quality from social big data, and thereby deliver better search results than its competition. Any of the other search engines or big incumbent IT players of the time could have done this, but didn’t. They focused solely on the information available within the web pages, not the information implicit in the connections between web pages. They seemed blind to the absolutely free and accessible data in front of them. And because of that blindness, they essentially ceded the entire future of search to Google. 

From the perspective of our model, Google is an example of applying new algorithms to new kinds of big data.


Google is a relatively new company in the dynamic IT industry. How about a traditional industry, such as retail? In retail, Walmart rules. Yes, as most Walmart case studies point out, better supply chain logistics have been an important advantage. But that kind of process advantage can ultimately be copied. A key sustainable advantage is the proprietary customer purchase behavior information Walmart gathers and uses to deeply understand consumer preference inferences. Walmart is the sole owner of the data and resulting insights, which gives it leverage with their product suppliers.

From the perspective of our model, Walmart is an example of mostly applying existing algorithms to new kinds of big data.

Goldman Sachs

How about the world of finance? Even with all of the ups and downs in the financial world, Goldman Sachs and a number of “quant” boutiques have stood out from a performance standpoint for many years. Why? Goldman and the quant boutiques more effectively applied sophisticated algorithms to wring every bit of possible predictive insight out of the data available. They are typically exploiting market data that is available to everyone else, they just have superior models and algorithms that are applied to the data. 

New algorithms applied to existing data.

Life Sciences, Pharmaceuticals, Biotech

Then there are the life sciences, pharmaceuticals and biotech. You win in the life sciences with R&D. You win in R&D by having a deeper understanding of very complex phenomena than the competition. And you do that by generating more experimental data points than the competition, and making better meaning from those data points. Massive parallelism (e.g., high throughput experimentation, combinatorial chemistry, genome sequencing) has enabled orders of magnitude more data to be generated, and advanced statistical techniques have enabled better predictions from the data.

A number of top tier pharmaceutical and many more smaller biotech companies have gained competitive advantages with these approaches. Those that did not aggressively take advantage of applying these new algorithmic techniques to newly available forms of big data ended up with underperforming product pipelines.

Putting This Into Practice

We can draw several important conclusions about the big data-as-a-disrupter model from these cases:

  • The model can potentially apply to just about any industry
  • The market value of just these handful of examples alone suggest big data-as-a-disrupter may be the most successful, but under recognized, business approach of the past decade
  • The model clearly extends beyond individual businesses -- it can have disruptive effects on entire industries


How can you put the big data-as-a-disrupter model into practice for your own business? Start by identifying potential opportunities. A good place to begin to explore possibilities is with the data. Think first about data that you already have access to that could be leveraged for new purposes. This may be data for which you have proprietary access or it may be more widely available data that has been under-exploited. Brainstorm without imposing any limitations on what kind of insights could possibly be derived from this data.

Then think about data that's currently unavailable to you, but could potentially be captured or generated. Identifying already accessible data or data that could be generated often starts with the internet, where massive amounts of behavioral information is continuously generated and captured. But consider other sources of information.

Good examples to consider are data generated by real time operating processes, by transaction processing systems, by IoT devices, by RFID-based systems, and by mobile devices. In general, data generation sources boil down to people, machines and instrumentation, as well as the environment, so be sure to brainstorm possibilities derived from all of these sources.

In any of these cases, the data available or generated may be truly massive, yet also highly noisy, so deriving meaning from the data can be a challenge (if it was simple everybody would already be doing it!). Coming to the rescue algorithmically are a variety of rapidly evolving techniques for making meaning from large amounts of noisy data, such as machine learning-based models, that you’ll need to stay abreast of and evaluate.

The big data-as-a-disrupter business model has clearly formed the basis for some of the biggest business winners of the last 10 years. Yet we've barely scratched the surface -- tremendous sources of data lay untapped and some of the most powerful analytic approaches have only recently been developed. The race is on: be the first-mover in unleashing the disruptive force of big data in your marketplace.