Here’s the rub: big data won’t do any of these things. Highly skilled clinicians and biochemists will cure cancer. Bad privacy policies and poor data security will destroy your privacy. And skilled business people will find ways to capture millions in revenues. Yes, big data will help. But the machines can’t do it alone.
Large enterprises are trying to catch the big data wave by buying the technologies (focusing first on storage capabilities, not analytics) and taking a “let’s store all the data we can until we figure out what it might be good for” approach. The myth that drives this behavior is that somehow, magically, big data will sift through all that information and produce gold. What drives this behavior is big companies know they don’t know what to do with that data and they can make the ROI numbers work by replacing other storage options, like tape backup.
SMBs don’t have the luxury of building data lakes and hoping. And that’s a pretty silly strategy anyway. It’s analogous to owning a library full of books you never plan to read. Big data absolutely is a cool new tool, but it really doesn’t change how you apply analytics to your business. SMBs can use big data, but only if they're highly pragmatic in their approach.
Mind Your Business
The trick to using data to drive business performance comes down to one key factor: understanding your business model.
Where are the key leverage points in your business? If you knew more about your customers’ preferences, what could you do about it? If you could monitor a key process, could you optimize it? Is there information you have that you could deliver to a customer or partner that would make your product or service more valuable?
Answering these questions will help you identify where big data might generate an insight that you can act on to create value. If you can’t think of an interesting question to ask that could lead to value creation, you don’t need any data, much less big data.
My best analytics projects have always started with no budget. No new technology, no new staff … just get started with what you already have. Once you have identified an interesting question, start trying to answer it with the resources you have on hand. Find the person in the business who knows and cares the most about the problem you’re working on. Find the most data-savvy person you can find inside your organization. Don’t hire a PhD data scientist just yet.
Start working the problem. See if you can find the data you need to answer the questions. See how far you can get with the analytics tools you already have. Exhaust the first line of analysis before you try to use exotic techniques. Why? Because you’re likely to learn that you found an answer without big data, you had the wrong question and needed to pivot, or you couldn’t get the data and need to instrument a business process so you can get it.
Rent to Own
So you had the right question but didn’t find the answer on the first round of analysis. Now's the time to bring in the big data magic. Before you go fill out the CapEx request for hardware, software and implementation costs, think “Cloud.”
Services like Amazon’s EC2 enable you to create your own analytic sandboxes, and you pay on a usage basis. You can fire up an environment to do some initial testing and turn it off when you’re done, paying just for the time you used. When your initial experiment is a success, you can scale that environment up to handle the full big data load. And when the project is completed, you turn it off. Capex becomes Opex, total cost for your project goes way down, and you can use a “fail fast” model to test many possibilities before funding an initiative through completion.
Need help? It's probably still not time to hire that data scientist. Because you defined your question well and have done the initial analysis, you’ll know enough to engage a very targeted consultant. Not a “Big Data Generalist,” but someone who has experience with the specific types of problems you are trying to solve. Options abound from big consulting shops to a graduate or doctoral student whose thesis is in your area of interest. Engage them on the specific problem and move from there.
Business is becoming a data-driven game. Even marketing departments are being run by data-driven behavioral economist types rather than the creative copywriters of a decade ago. This shift isn’t about any given technology, it's really about a mentality. Focus on problems that matter, define measurable methods for understanding and managing those problems, and use analytics (big or small) to create value by solving those problems.