By now everyone from your kid’s baseball coach to the Avon lady representative can give you an earful about how (big) data and analytics lead to better decisions. But if that’s the case, why are so many of us seeing so many lousy offers?

A few years ago we might have been able to argue that retailers, and other marketers, thought that big data was just a bunch of hype that didn’t actually lead to better returns. But we’ve come a long way since then.

The Value of Shopper Insights

A recent survey, conducted by RetailWire and commissioned by big data analytics provider Alteryx, reveals that 73 percent of retailers consider shopper insights to be very important or essential to the performance of the departments in which they work. In addition, 76 percent think leveraging shopper insights is important to the performance of the company as a whole.

The same survey indicates 81 percent of respondents are collecting data relative to things like demographics, social media activity, online purchase and browsing data, loyalty program activity, customer interaction/complaint data, 3rd party research data, in-store movement behavior, mobile purchase/browsing data, media and entertainment preferences and so on..

So if we’ve got the data and the interest, what’s the problem? 

Only 16 percent of the 350 retailers and brand manufacturers surveyed consider themselves to be “experts” in deriving shopper insights from their data and using it to drive retail decisions.

That’s an expensive knowledge gap. And since nature abhors a vacuum, technology and solution providers are rushing to fill it in different ways.


Defining the Enterprise Data Hub

We’ve been hearing Hadoop provider Cloudera talk about its Enterprise Data Hub strategy for nearly a year now. To many, it has seemed a bit like a “pie in the sky”.

It won’t anymore. Today, there’s proof in the pudding.

Rony Sawdayi, vice president of engineering at Shopzilla told us last night that they have deployed Cloudera’s enterprise data hub to complement their existing Oracle enterprise data warehouse (EDW). Though it will likely be leveraged throughout Shopzilla, it is now being used to help power Connexity, a platform that enables brands and agencies to learn from and connect with shoppers at any buying stage wherever they are.

More specifically, Connexity works much like a super-smart media buying platform that leverages a number of its unique assets which include: Shopzilla’s insight on buyer behavior that has been refined over the last 18 years.  

Data from Shopzilla's global portfolio of retail websites which connect more than 40 million shoppers with over 100 million products from tens of thousands of retailers. and its now unlimited capacity to process and deliver new insights on millions of page views and ten billion ad requests daily, reaching over 100 million unique visitors and gaining valuable insights in hours or minutes instead of days. To learn more about how Cloudera and Shopzilla worked together, go here,

Talk About Actionable Insights

For those who are more interested in practical examples, consider that in as little as a few milliseconds Connexity can score the probability of someone clicking on an ad and converting or not.

Learning Opportunities

Ponder the case of a retailer who wants to appeal to sports enthusiasts. Connexity can find them on the web detailed down to the particular sport they participate in, where they live and play, their range of income, where they are physically located at this very moment, what website they visited last, where they spent time and what they looked at … and so on.

It’s with this kind of information that Connexity can predict, with some degree of certainty, an individual’s next action. A “buyer” will then have to decide, within a few milliseconds, whether an ad should be served up, or not. Remember the decision has to be made before the prospect lands on the webpage.

For anyone who was wondering why enterprise data hubs are needed or why data scientists make so much money, wonder no more.

How Looker Helps You Glean Insights

Sure, companies like Walmart, Amazon and Google might all employ armies of data scientists who are singularly focused on getting customers to click on ads or on flashing buttons that say “Buy Now”.

But not every company has that luxury.

Instead some employ a single data scientist or a few business analysts who have stepped up to the plate to help their companies get better results in the name of data.

And while the efforts of these gallant data gurus are admirable, here’s what usually happens, “I spent most of my time writing queries for users,” said Ty Wilson, a senior business analyst at CustomMade Ventures, an online matchmaker connecting Buyers who want one-of-a-kind Creations with professional and passionate Makers.

Halumeer “Hal” Ali, a business intelligence specialist at Frank& Oak, a custom men’s retailer, probably knows just how he feels, “ I was interrupted so often that I didn’t have time to work on important (data driven) projects like fraud prevention,” he said.

But those days are now gone for both of them because their companies have now implemented Looker, a data discovery tool which uses a proprietary language called LookML, that allows users to write their own analytic queries. It's a SaaS solution that takes a few weeks to implement and train and it turns everyone into a data scientist.

OK, not really. But it does let guys like Wilson and Ali work on tough analytics projects that need their best skills because more users are now writing their own queries.