flowing water.jpg

Two years ago, no one seemed to know what to do with big data.  Now, vendors are offering solutions that are intuitive enough to provide real business value, enable even non-data scientists to perform real analysis and have ushered in a new era of novel applications like data-driven design.

Big data is maturing — evolving from a frothy, over-hyped concept to a source of actionable insight.  It has captured the imaginations of companies of all sizes --including Ford, GE and Universal Music Group -- given a new voice to customers worldwide and has helped to create business value at a blistering pace.

Those were my main takeaways from the GigaOm Structure Data conference in New York City this week.

Like Water for Data

The most important thing to remember about big data is that it's a potential source of deep insight — and many companies still haven't fully leveraged its benefits.

New technologies, however, are making it easier and easier to derive value from that ever flowing data.

I like what ClearStory Data has developed over the past 18 months or so, which seems to be a version of Excel for the Enterprise with sharing capabilities.  At the conference Wednesday, the company unveiled new data harmonization, in-memory data processing and user model advances for faster data analysis across greater internal and external sources. The company claims the advancements "allow organizations to access, converge and harmonize dozens of disparate data sources on the fly for broad insights without requiring IT experts and long data manipulation cycles."

CEO and founder Sharmila Mulligan said the company's mission is  "to dramatically simplify accessing, exploring and analyzing data."

The big challenge I see with the product is getting various departments within an enterprise to write connectors to the ClearStory Data so the information can be curated in a useful way. The challenge isn't actually  writing the connectors, but knowing what should be connected and the best way to add texture to it.

Data-Driven Design

More organizations are using data to improve design. It's proving to be the fastest way to successfully test and implement new designs. At the conference Wednesday, Geoff McGrath, managing director of McLaren Applied Technologies, said the same design techniques used in automotive design can be applied to many other industries and products, from bicycles and baby strollers.