There are some pretty safe bets to make around big data in 2014, so we’re going to make those and leave it to some of the leading big data innovators -- Cloudera, Hortonworks, MapR, Microsoft, Pivotal and SAP go out on the limb. 

Our Predictions

  • By the end of Q3, “big data” will be known simply as “data.”
  • “Hadoop on the Cloud” will be on the table in enterprise strategy discussions. The business will want in on the conversation.
  • There will be acquisitions and consolidation in the Hadoop ecosystem.
  • More workers will be leveraging “big” data and the new breed of analytics to make small, as well as big, decisions.
  • Security becomes as important as big data, cloud, social and mobile

What the Vendors Say

Enough from us, we invited six industry leaders to make up to five predictions for 2014. Here are their answers:

Amr Awadallah, chief technology officer, Cloudera

  • 2014 will be the moment of truth for big data -- Customers are getting fed up with all the buzz around 'big data.' They want to see what the technology can do for them -- right here, right now. Cloudera believes the Enterprise Data Hub will rise to this need by allowing companies to consolidate all their data into one system that can run multiple workloads to address virtually any enterprise data need.

  • Stronger Hadoop security -- Hadoop security took a big step forward in 2013 with the release of the Apache Sentry project. We expect to see further innovations in security in 2014 to enable Hadoop to rival legacy data solutions.

  • Big data moves to the cloud -- In 2014 we expect to see a big shift as heavy, backend data systems and applications begin migrating into the cloud. For the first time, enterprises are now comfortable enough to have their core data (their blood) stored outside the company’s walls. Not all industries will make the jump in 2014 (e.g., financial services, healthcare and other industries with stringent security considerations), but we expect enterprises in retail, telecom, automotive and other verticals to make the switch this year."

Quentin Clark, corporate vice president, Microsoft

  • We will start to address the 'last mile' problem in big data. While the last few years have been about maturing big data tools, I think we will start to get at what I call the last mile problem. We’ve put the infrastructure in place for high-end specialists to get value from big data, but how do we put this in the hands of everyday users and let them ask their own questions and get their own insights? Many will agree that the value of big data is only realized when a business takes action -- and people are a big part of taking action. What you will see in the next year is that tools like Excel will be connected to big data back-ends so people can get the power of big data in the tools they use every day.

  • We will begin to see a large percentage of big data and Hadoop solutions include data that is stored on premises and in the cloud. We have learned that so much of the value of big data comes from the diversity of the data. But it is not always diversity in new data signals. Actionable big data comes from data sources out of traditional business applications, out of new signals and out of external signals like industry partners, government data and even other industries.

  • The industry will begin to shape around data models that will help organizations maximize the value of their data. The questions and insights we can get will deeply join the conversation in the coming year. Composing the right variety of data, at the right scales and on a speed-of-business cadence provides the right environment, but the models and questions we ask are the other important component. To get to the right insights and impact -- to achieve actionable big data -- we have to be asking the right questions over the right perspective on the data. We are at the early stages of the industry shaping around models that help every industry, every company get value from big data / Hadoop, and then start to maximize that value."

Steve Lucas, president, SAP Platform Solutions at SAP

  • Big Data will force a new storage architecture to emerge: As the conversation around big data grows beyond ‘data warehousing’ and towards integrating it into core business processes, there will be a growing recognition that big data is a velocity problem first and volume problem second. This will drive the conversation towards technologies that accelerate the end-to-end process to acquire, analyze and act on big data. Furthermore, an increasing concern about the cost of storing massive data sets, and a continued lack of expert resources with NoSQL technologies like Hadoop, will drive a new emphasis on the cloud friendly solutions. However, this emphasis will also force companies to pay attention to advancing questions around data privacy and security, and will force a new infrastructure / roadmap to emerge that can address all concerns.

  • The cloud will develop beyond storage to become an innovation platform: The days of building a business case to move to the cloud based on total cost of ownership are over. Companies will increasingly look to invest in the cloud as their innovation platform -- as a way in which they can extend their existing infrastructure, enable new processes and achieve new insights that enable them to run their bus in an entirely new way -- not just do the same old thing using a different delivery model.

  • More looking forward, less looking back: Organizations of all shapes and sizes have trumpeted the value of knowing your data and using it for predictive purposes. 2014 is the year that Business Intelligence makes the leap, with enterprises truly embracing and using their historical knowledge to make predictive leaps -- looking forward and employing predictive analytics to 'skate where the puck is headed.'"