We talk often about big data and big data technologies. However, in all the discussion around the best deployments and the best enterprise strategies, the human element in the equation is often overlooked. In the introduction to the AIIM Executive Leadership Council’s examination of this problem, John Mancini and Thorton May argue that this needs to be addressed in order for enterprises to achieve an acceptable level of ROI with their big data deployments.

Yin-Yang, Business versus IT

Drawing parallels between the Yin-Yang balance models of Chinese mysticism, they suggest that if enterprises are to successfully manage big data and develop big data strategies, they must solve the big data yin-yang conundrum.

The conundrum is simply described in the white paper "The Big Data Balancing Act -- Too Much Yin and Not Enough Yang" as the often conflicting needs of technology and business.

This is not the first time we have come across the IT versus business needs conflict, but by evoking yin and yang, May and Mancini point the way forward to a way where both sets of needs are met and big data and related analytics becomes a positive differentiator rather than a negative source of enterprise conflict.

That there are supply issues around personnel working in big data has also come to our attention in the past. In recent months, for example, IBM has opened a number of "analytics colleges," to educate a generation of data scientists that are thin on the ground at the moment, a situation that makes big data problems even more difficult to address.

And here we are back to the conundrum again. If technology is the yin, the paper asks, where is the enterprise yang? The simple answer is that it is the business side of big data -- "the action as opposed to the analysis, and the methods and people required creating this harmonic balance."

The conundrum, then, in big data terms is this: how do enterprises put information extracted with big data technologies to beneficial business use? Current big data technologies talk of predictive analytics, Hadoop, MapReduce and others. But these are only technologies; what human elements will make it work?

The Work Entailed for the Enterprise

To resolve this issue, they say, there needs to be new thinking in the enterprise. Enterprises need to work out entire frameworks for its deployment that takes all the elements needed and successfully pulls them together. There are four areas in particular where there are tensions that need to be resolved:

  • The Data Scientist
  • The Data Entrepreneur
  • New Ways of Working
  • Putting Data to Use

1.The Data Scientist

There is a shortage of data scientists -- this we know. Citing Diego Kablan, founding director of the Master of Science in Analytics at of Northwestern University, May and Mancini point out that some of the skills need to be successful here are IT, science, business and analytics.

But almost by definition, those that follow this path must be skewered towards IT. The reasons for this are obvious; to be successful here you need to know computer languages too, including PHP, Python and to be able to distinguish Javascript from XML.

big data skills balance.jpg
Big data: skills balance
To have two of these languages is good, three is very good, but four is rare and commands a price tag to match. However, May and Mancini ask, while these skills are necessary, do they all have to be concentrated in one person?

Citing Michael Schrage of MIT Sloan School’s Center for Digital Intelligence this time, they suggest that businesses need to move from being data-driven to being aware of what needs to be done and what can be done.

Current demand is for a data scientist that is expert in all the technology issues surrounding big data, but with a whole bundle of business skills on top of that.

However, this may not be the way to go. To have the level of IT knowledge and knowledge of what is needed for the business in the same person is not realistic.

Balance is required here. While your enterprise needs people that have the necessary IT skills, they also need business people that understand big data projects. This, they say, requires a Data Entrepreneur.

2. Data Entrepreneur

The data scientist is implicitly methodical, some might say mechanical, and concerned with technology and analysis -- that is their focus after all.

We have looked at the enterprise and seen that while data scientists have a core skill set, they need to be separated from the business of business. Business people, however, also need to understand big data and what can be hidden there, even if they don’t necessarily have the skills to go mining for it.

On top of this, May and Mancini suggest that data scientists can set up the data mining, but may not necessarily be in a position to recognize something that can be turned into revenues. This is where the business element comes to the fore.

big data skills set.jpg
AIIM:big data skills focus
To cross the divide, they say, enterprises need a data entrepreneur. That is someone who knows exactly what the data scientist knows but can’t necessarily do it themselves, but also knows a good business opportunity when they see it.

This data entrepreneur is an information professional in every sense of the word, working across disciplines such as social media, content management and governance, but never really an expert in it all.

But there is much more to a data entrepreneur than their skills in business, technology and data. They also need to demonstrate the true entrepreneurial skills of innovation, creativity and desire to achieve excellence through the generation of new ideas.

3. The New Way of Working

Having resolved two of the stress points in the enterprise, the next thing is to establish is an effective and disruption-free way of pulling pulling big data analysis into the enterprise.

Again Michael Schrage is cited. He introduces the idea of  bringing hypothesis-testing into the enterprise as a way to formalize big data analysis.

The difference here to current practices is that it  moves away from a scenario where technology is used to identify problems, to a scenario where it is used to solve problems once they have been identified.

This approach takes its lead from the business side of the house. Business people identify problems that are seen as business problems, while the IT side of the house can provide insight into those issues once the problems have been outlined. You can see here again where the data entrepreneur is going to fit in.

The “hypothesis” way of working is one where a possible explanation is offered for a particular issue and then dissected to see if the explanation, or hypothesis, actually works.

Establishing what hypothesis to create and test is the most difficult aspect of thist, but also the most important in this new way of working. 

4. Putting Insight To Use

The last part of all this is to differentiate between action and analysis and to ensure that the results that come out of your analysis become actions rather than just results.

This is not necessarily a complex and time consuming process once the analysis has been done. May and Mancini cite the example here of elderly shoppers. If the hypothesis is that elderly shoppers buy more, it may take some considerable time and information to work that out, but once it is known for definite then the business decision to provide products that appeal to elderly shoppers would be a rapid and common sense one.

Here, again, the data entrepreneur bridges the two sides of the business by postulating a number of scenarios and bringing them to the IT side of the house for analysis.

The deployment of insight moves big data analysis beyond simply creating interesting results, to being truly beneficial to business.

Like all yin and yang situations, balance is not complete with only one half of the circle considered. Currently there is a tendency to think of big data in terms of technologies only, rather than technologies working in a business environment.

To do so, the paper concludes, is to negate many of the possible benefits that big data and its analysis can bring to the business, a fact that is not going to auger well for investment in big data.

Enterprises need to see returns on investment and by offering a business model that works with big data may be the way of demonstrating ROI. It also just makes sense. There is a lot more to this argument than this summary so if you are interested in pursuing this further check it out here after free registration.