Just ask a statistician. Big data is simply anything that won't fit in Excel.
This was the conclusion of a panel of experts at the eMetrics Summit this week, and what they meant was, if one uses Excel to process and analyze data, it's not yet of the big data variety (Microsoft may be inclined to disagree with this comment based on its latest research). It's only when the tools normally used to perform needed analytics are no longer enough that it officially becomes big data, a group of data scientists and statisticians agreed at eMetrics.
People + Process Over Tools + Technology
The eMetrics Summit has been running for over a decade, and it brings together practitioners, consultants and vendors in the marketing data, Web optimization and digital analytics worlds. As these worlds collide, and companies decide they need to crunch ever larger amounts of information to find critical insights, the big data bonanza has struck.
"We think about our customers as much as we do digital data," Michael Ebeid, McKinsey & Company senior expert for customer and shopper insights practice said as he gently pooh poohed the big data concept.
It's less about how big the data is," he said. "It's really about depth and breath and how useful is it for things like predictive modeling."
Big data is simply whatever doesn't fit into Excel, Stephane Hamel, a consultant with a company called Cardinal Path, and author of the Online Analytics Maturity Model said at eMetrics. "Its getting out of our comfort zone and using other tools," he said. Big data starts when you have to use any other tool besides a baseline BI type system or, of course, Excel, Matthew Wright, a data scientist at Hewlett Packard said.
The problem, Wright said, is when a customer right away starts talking about what analytics tools they are using. Too often, people tend to focus on the tools and not the type of problem they need to solve, he said. Big data has become so much of a buzz word, that the underlying importance of what it once stood for is simply unrecognizable as anything useful.
Buzz words don't matter," Wright said. "Unless big data comes about as a result of a natural conversation with a client, it's probably not necessary. The whole concept of big data tends to skew the industry to focus on the tools."
Excel (pictured), SPSS and SAS are common tools that can do much of what people consider big data analytics, an eMetrics Summit panel concluded.
The Data is There, It Just Needs a Purpose
A big misconception about mining data for new insights is that some new tool is actually needed to find it, Hamel said. The data is often already on hand, and companies simply need to think harder about what it is they want to do, and how that data can help them.
To that end, marketing and IT teams seem to be on nice little collision course, Hamel said, and that could mean the start of more detailed collaboration on actually solving problems instead of prattling on about buzz words.
This extends, of course, to the enterprise, and for those looking to invest in new big data tools, there is lots of inertia against risk. Focus investment on what makes money, Wright said. Pick a few small projects as pilots, and build the business case that way.
It's harder to get tied down to a particular architecture or hierarchy too early that way, he said. The constraints built into large enterprises force you to just do a few projects that you can focus on and learn from.