chelsea piers

If you’ve followed the big data hype long enough, you know that every year -- since 2011 -- was supposed to be big data’s big year. And that at the end of every year since, the pundits have said “It didn’t happen this year. Next year will be big data’s big year.” 

The analysts, of course, have been more cautious than the marketers; in fact, late last year Gartner said that it will take longer (five to 10 years) than they thought for big data to crawl out of its trough of disillusionment and onto the plateau of productivity.

They cite two reasons for this: first, adoption of big data tools and techniques is happening ahead of learned expertise and maturity/ optimization (which if correct is a prescription for disappointment) and second, that enterprises have been unable “to spot big data opportunities” and “formulate the right questions and execute on the insights."

Tom Davenport, in his Harvard Business Review webcast, Big Data at Work: Dispelling the Myths, Uncovering the Opportunities, isn’t as pessimistic. He said 2015 may be the year big data crosses the chasm.

While both Gartner and Davenport are great advisors, we thought it might be interesting to see what people who have their feet on the street are saying. GigaOm’s Structure Data Conference in New York City gave us the perfect opportunity to do so.

Cloudera’s Take

“Big data adoption is accelerating rapidly,” said Alan Saldich, vice president of marketing at Cloudera. “While a few years ago customers were asking 'What is Hadoop?' and the next year they were asking 'What can I do with it?' they now have business strategies around it.”

And those strategies, mind you, aren’t just about off-loading data from expensive, proprietary hardware and dumping it all into Hadoop where storage and processing is at a fraction of the cost, but about discovering and applying insights that weren’t conceivable in the past.

Hortonworks is Bullish on Hadoop

Rob Bearden, CEO of Hortonworks is bullish as well. Standing before a packed house at the conference he told the audience that the Enterprise Open Source Hadoop provider would be a $1 billion company in the next four years.

We asked David McJannet, the company’s vice president of marketing if 2014 would be big data’s big year.

We’ve seen significant growth in all the leading indicators of our business -- from number of customers to rate of adoption -- that make it clear that interest in Hadoop has moved beyond investigation. The early majority has moved into real production deployments for their big data projects, and as a result, the big data market is accelerating at an unprecedented rate.”

Hadoop Use is Growing Among MapR Customers

Jack Norris, chief marketing officer at MapR offers a similar take, “We're seeing Hadoop adoption rapidly accelerate as customers are realizing revenue gains, risk mitigation and cost reduction. Customers are also expanding their use within their companies. More than 80 percent of our customers expand their Hadoop cluster within the first year.”