As someone who spends a lot of time talking about the power that data insight has to transform businesses, I have to admit that the truth hurts. 

And the truth is that despite myriad advancements in modern big data platforms and technologies, most companies still lack data insight.

This is something of a good news/bad news situation. The bad news is obvious: Despite all of our efforts and investments, data insight remains elusive. 

The good news is that we’re not bogged down by complex or, worse yet, unknown problems — problems to which we don’t yet know the solution. Rather, it’s the same old problems of yesterday that still conspire to keep us from achieving the level of insight we desire. Understanding them, and taking the necessary actions to rectify them, is the key to making data insight a reality. 

Let’s take a closer look at five of the most common reasons most companies still lack insight.

Poor Data Management

I’ve written in the past about the importance of data management fundamentals, but it bears repeating when talking about the roadblocks standing between businesses and insight. 

In an era when companies have to manage larger and more complex volumes of data than ever before, sound data management is imperative. If you don’t have best practices in place to simplify database administration and optimization, deliver proper data modeling, manage metadata and ensure data quality, you will not achieve your desired level of insight regardless of how much you invest in big data tools. 

Without proper data management, you might get some insight, but the process will be slow and arduous, the findings incomplete, and the skepticism from stakeholders rampant.

Lack of a Semantic Layer and Metadata Management

Semantic layers were devised to assist businesses in better understanding and prioritizing the importance of data. Without a semantic layer, you really can't have a consistent view into things like how you do analytics or how you define customers. Lack of a semantic layer is why you see so many companies with conflicting definitions of what constitutes a customer and/or prospect within their own systems. 

Drilling a level deeper, reliable and properly managed metadata — or data about data — is a must in order to truly understand how data can and should be combined, regardless of where it’s coming from. Without a semantic layer or proper metadata management, the difficulty level associated with achieving data insight skyrockets.

Bypassing Data Modeling

In the age of the NoSQL database, data modeling has, unfortunately, become somewhat taboo. Many developers now feel as though the agility afforded to them by NoSQL technologies has eliminated the need for data modeling. 

That might indeed be the case when it comes to answering smaller, finite questions within a contained data set. But when you’re attempting to answer bigger and broader (and usually more important) questions that involve marrying different data types, the basics of data modeling are essential.

Data modeling enables you to normalize data, create commons dimensions across data, map data to customers and ultimately generate reliable reports.

Confusing Visualization with Analytics

Let me preface what follows by stating how much I value and appreciate what visualization brings to the table. Visualization has fundamentally altered the BI landscape for the better by allowing analysts to state their cases in a more compelling manner. And as we’ve discussed in the past, insights are only valuable to the extent that actions are taken against them. 

So, in that sense, visualization has greatly advanced the analytic cause. But make no mistake, visualization and analytics are not one and the same, even if many people make the mistake of thinking they are.

If visualization is the exterior finish of a building, analytics is the foundation and structure. The exterior of a building can look fantastic, but the building might still fail inspection if the structure is faulty. Similarly, if not grounded in proper analytics, data visualizations crumble once you dig below the surface — or worse yet, once you take action on the insight they purport to deliver.

If you constantly feel like your data insights don’t match up to your market and sales realities, and your business results bear that out, then it’s time to start asking tougher questions about what you’re seeing. Are the underlying analytics sound? Is the supporting data valid? Were you asking the right questions to begin with?

Visualizations are valuable, but without data and data analytics practices serving as the foundation, they do not deliver insight.

Putting Out Fires and Starting Science Projects

Like anything worthwhile, it takes time and energy to obtain insight from data. And unfortunately, far too many companies are wasting far too much of both. Wasted time and energy usually takes one of two forms.

The first is time spent putting out fires: "This database isn’t properly optimized." "That application went down and we’re not able to quickly recover it." "Remote workers can’t seem to connect." 

The time you spend putting out fires is time not spent innovating and uncovering data insights. If that sounds like your organization, then it’s time to outline a plan and invest in the technologies needed to make your IT team more productive and keep those fires at bay — or better yet, keep them from occurring in the first place.

And when we’re fortunate enough to be free of fires, we often spend far too much time on science projects — our latest attempt to impress the boss and advance our career by latching on to the latest trend. I’ve been guilty of this many times myself. 

We’re all eager to sink our teeth into something splashy. More often than not though, all we wind up doing is sinking time, energy and money into a project with little tangible value. Time is our most precious resource, and if we truly want to achieve data insight, we need to be wiser about how we spend it.

It’s Not Too Late

I started by talking about bad news and good news. So here's the really good news: it’s never too late. 

No matter how you’ve operated in the past, it’s never too late to change your approach. It’s never too late to decide that it’s time to get serious, do the homework and make data insight a reality.