We talk often about big data and big data technologies. However, in all the discussion around the best deployments and the best enterprise strategies, the human element in the equation is often overlooked. In the introduction to the AIIM Executive Leadership Council’s examination of this problem, John Mancini and Thorton May argue that this needs to be addressed in order for enterprises to achieve an acceptable level of ROI with their big data deployments.

Yin-Yang, Business versus IT

Drawing parallels between the Yin-Yang balance models of Chinese mysticism, they suggest that if enterprises are to successfully manage big data and develop big data strategies, they must solve the big data yin-yang conundrum.

The conundrum is simply described in the white paper "The Big Data Balancing Act -- Too Much Yin and Not Enough Yang" as the often conflicting needs of technology and business.

This is not the first time we have come across the IT versus business needs conflict, but by evoking yin and yang, May and Mancini point the way forward to a way where both sets of needs are met and big data and related analytics becomes a positive differentiator rather than a negative source of enterprise conflict.

That there are supply issues around personnel working in big data has also come to our attention in the past. In recent months, for example, IBM has opened a number of "analytics colleges," to educate a generation of data scientists that are thin on the ground at the moment, a situation that makes big data problems even more difficult to address.

And here we are back to the conundrum again. If technology is the yin, the paper asks, where is the enterprise yang? The simple answer is that it is the business side of big data -- "the action as opposed to the analysis, and the methods and people required creating this harmonic balance."

The conundrum, then, in big data terms is this: how do enterprises put information extracted with big data technologies to beneficial business use? Current big data technologies talk of predictive analytics, Hadoop, MapReduce and others. But these are only technologies; what human elements will make it work?

The Work Entailed for the Enterprise

To resolve this issue, they say, there needs to be new thinking in the enterprise. Enterprises need to work out entire frameworks for its deployment that takes all the elements needed and successfully pulls them together. There are four areas in particular where there are tensions that need to be resolved:

  • The Data Scientist
  • The Data Entrepreneur
  • New Ways of Working
  • Putting Data to Use

1.The Data Scientist

There is a shortage of data scientists -- this we know. Citing Diego Kablan, founding director of the Master of Science in Analytics at of Northwestern University, May and Mancini point out that some of the skills need to be successful here are IT, science, business and analytics.

But almost by definition, those that follow this path must be skewered towards IT. The reasons for this are obvious; to be successful here you need to know computer languages too, including PHP, Python and to be able to distinguish Javascript from XML.

big data skills balance.jpg
Big data: skills balance
To have two of these languages is good, three is very good, but four is rare and commands a price tag to match. However, May and Mancini ask, while these skills are necessary, do they all have to be concentrated in one person?

Citing Michael Schrage of MIT Sloan School’s Center for Digital Intelligence this time, they suggest that businesses need to move from being data-driven to being aware of what needs to be done and what can be done.

Current demand is for a data scientist that is expert in all the technology issues surrounding big data, but with a whole bundle of business skills on top of that.

However, this may not be the way to go. To have the level of IT knowledge and knowledge of what is needed for the business in the same person is not realistic.