Big data is out, small, smart data is in. That’s the word from the folks who have tried to tame big data and failed.
Big data is unwieldy…“useless, or even more than useless because it can distract businesses and consume a lot of resources for no value return,” according to Mike Fauscette, IDC analyst and Group Vice President. In a recent series of articles entitled Transforming Data in Action, Part I and Part II, Fauscette primarily lays the blame with the difficulty of finding "information needles hidden in Big Data haystacks."
Andy Mulholland, vice president and principal analyst at Constellation Group and formerly Global Chief Technology Officer for the Cap Gemini Group, generally agrees. But whereas Fauscette highlights the challenges of filtering data and determining its relevance, Mulholland focuses on the difficulties associated with making sense of the many pieces of relevant, but disconnected information. In a recent article entitled "Internet of Things; Requires Big Data to be turned upside down to become Smart Data," Mulholland writes, “We need to start working on how users will gain from enterprise Big Data by delivering it … in new graphical formats.”
I recently spoke with both analysts, who provided insights into how workers in the future will be able to exploit enterprise information in intelligent ways.
Big Data is Useless
It wasn’t long ago that big data was the next big thing. Using a combination of technology and methodology, big data promised to take huge data sets and expose correlations that highlight business trends. The productivity gains from big data were going to revolutionize business by allowing managers to make intelligent decisions based on real operational data. And while specialized business analysts have been able to exploit it at a macro level, big data has failed to provide individual workers with the insights they need to take action on a daily basis. On the contrary, the distraction big data brings to the workplace is the reason Fauscette calls it "useless."
Long Live Small, Smart Data
There is nothing wrong with big data per se; but it’s not actionable for individual workers. What workers need is not big data, but rather, relevant data presented in smaller and smarter chunks.
According to Fauscette,
Big Data can be made smart data if it can be made smaller, by transforming it to become contextual, relevant, and delivered to the right person at the right time, in the right format .… Smart data is data in context, in the ‘right hands’ and relevant to some issues, activity, problem, etc.”
Making Data Smart is Not Easy
But even that’s not enough. Because even when "the right set of data is available to the right person at the right time," it’s tough to make sense of it all, to say nothing about turning the data into actionable activities.
Fauscette identifies the difficulty in individualizing information for each person as the missing piece. For example, Fauscette points to Amazon’s recommendation engine, which suggests items to buy based on what other people bought, rather than on a deep understanding of the individual. In a work context, automated individualization is not practical. People need to individualize their own experience by "slicing and dicing" information from disparate sources to extract business insights. They need to be able to view combinations of relevant information according to the most appropriate context for a given situation.
Mulholland sees the problem as the inability to present combinations of relevant information in a graphical format that is intelligible to the end user.He said there's a need to package data to, “extend our human senses with increased inputs that allow our personal experience and knowledge to be applied to evaluate and decide on responses.”
Both analysts are right. Regardless of whether you call it small data, smart data or something else, the way to conquer big data lies in pushing information from multiple applications and cloud services to individual devices in a single window mash up, in a way that presents information in easy to consume, contextual chunks. By visualizing the information in easy to understand views, people will be able to identify business insights and transform them into actions that move the business forward.
Let’s take a look at an example -- that of a sales executive. As part of a typical sales cycle, the executive needs to stay on top of simultaneous business developments that manifest through ERP, CRM and salesforce automation systems, as well as other applications and cloud services. To be actionable, the sales executive needs to view filtered updates from the different services: updates that focus on the specific customer account as well the people working that account. Plus, all those updates need to be presented in a common format so they could be easily understood. This is quite a challenge in a world where workers and organizations use applications and subscribe to cloud services from different enterprise vendors, such as Salesforce, SAP, Netsuite, Oracle, Microsoft and others.
While a lot of this sounds like rocket science, we aren’t that far from seeing some progress in the marketplace. Important advancements in federation services and context-awareness, together with a growing awareness of the underlying challenges make this an area on which to keep a close watch.