Customer Experience, Big Data: Just Because You Can Measure It, Doesn't Mean You Should
Big data is hotter than ever, but are we losing sight of our goals by getting wrapped up in the tool?

Big data is in every conversation, blog post and business strategy -- and rightfully so. Any interaction in social or digital media results in data, and each day 2.5 quintillion bytes of data are created.

There’s enough data on just about everything, and while you can use it to answer any question, data alone isn't going to become actionable insights -- you need to do that. When applied without thinking, simply measuring anything and everything that’s available can lead to poor conclusions and disastrous results. In other words: just because you can measure it, doesn't mean you should.

A few weeks ago I came across Simon Terry’s great post “Don’t confuse the tool with the result,” challenging us to be honest about whether we are still pursuing the goal -- or getting stuck on the tool we’re using to reach the result. Sometimes, we get so enamored with tools we build for ourselves, that we start to measure our progress by our proficiency in -- or compliance with-- the tool itself.

Wrangling big data into focused insights is hard to do, and the temptation is to just start measuring in the hopes that a result will emerge. We must fight this temptation -- Simon highlights that big data analysis must be driven by strategy and desired results, or it won’t be effective. By doing “big data” for big data’s sake, you’re confusing the tool with the result.

Online communities present a perfect use case for big data -- data-driven decision-making can help us make them healthy and engaging by understanding interactions between people and anticipating their needs. Focus on the outcome; use data to help solve problems instead of looking for problems to solve with your data.

I had a conversation with Simon to get his views on big data and how focusing on the results might apply to managing communities with big data. Here are a few notes to keep you on track:

1. Know Why

Know what you’re going to measure and why. Your business strategy should guide you on what metrics and indicators are the right results. Those are the measures that you want to see growing in the community, and represent where value should be created.

While in a customer community it’s tempting to measure “likes” and number of active members, it’s much more useful to understand why people “like” and join and how it’s trending over time. Understanding why helps you find the levers to turn the insight into action.Have an initial hypothesis as to the next best action you can take, and work to prove or disprove it. This process enables you to tie things you’re measuring to your objectives, ultimately helping you connect your actions to business value.

For example, if your goal in the customer community is to delight customers by providing better support, you should measure things such as:

  • Amount of time it takes for each question to get answered -- and how it trends over time
  • Number of valuable answers per question -- and how it trends over time
  • Number of self-help articles that a customer can thumb through
  • Number of questions with answers vs. number of questions without -- and how it trends over time
  • Change in customer satisfaction scores
  • Incremental revenue that can be attributed to all this customer delight

By having a clear goal and crisp metric that relate to this goal, you will show business value and be able to take more meaningful actions with your community.

2. Don’t Apply Metrics from Unrelated Fields

Working with employees and customers in a community is a new way of working. Just because it’s on the Internet, doesn’t mean that you need to measure it like you would a website.When the results you want differ because you are looking for different behaviours and using different strategies, you need different measures.

In our move from pre-social technologies to social technologies, we forgot to switch our measurement. While reach is important (getting people there is the requisite step to everything that happens after) -- it’s not everything. Value creation that impacts your strategy is much more important.This means that expectations need to be managed.Often it is critical to help the business to understand the new domain and the right measures to get value in line with the strategy.

3. Understand Context

Just looking at numbers, without understanding context, will yield funny results at best, and bad decisions at worst. For example, correlating the volume of tweets to the degree of destruction by Hurricane Sandy is flawed, because it doesn't account for communication towers being knocked out. If relief efforts relied on data without context, relief wouldn't have made it to New Jersey.

Learning Opportunities

Similarly, when your community is in a state of major upheaval, engagement numbers may be up, but it’s not the kind of engagement that moves the community forward. Understanding context only comes from having had the experience to make sense of it. This experience also helps us discern between cause and effect, correlation or simple coincidence.

The data does not know what else may have been going on in your organization that is not captured.This is why machines will never be able to fully place information they are seeing in our human context -- it’s a uniquely human exercise that can only be carried out by humans. When interpreting data, pay attention to prior experiences, and don’t discount your “gut” feeling and your domain expertise. Make sure to understand qualitative anecdotes as well as quantitative signals, as not all activity is created equal.

4. Look at Trends

The beauty of big data is that you can track trends over time to help refine your achievement of the right results. You can also discover trends that you may not have anticipated. Knowing what you’re looking for is critical, because it helps you focus around the problem you’re solving. However, by ignoring things outside of your problem definition, you are potentially ignoring meaningful signal. Think of all the information at your disposal as this modified Johari Window:

  • What you know: just that -- what you know
  • What you know you don’t know: your hypothesis that you are testing or the question you are answering. This is what comes to you when you query the data.
  • What you don’t know you don’t know: all the other stuff that, when discovered, may or may not be useful to you.

A lot of data lives in this third section. You can’t possibly make sense of all of this data, so you need to know what you’re looking for. However, you also need to have a mechanism in place to identify things that are important but you haven’t specifically queried for.

5. Do Something

None of this data is meaningful until you use it to create insights, and insights are meaningless until you take purposeful action to achieve the outcome you were planning for. When studying health signals and trends in your community, ask yourself: “what does it all mean for the results I need, and what am I going to do about it?”

If you’re seeing that there’s increased activity in your community, dig in to see what’s causing it. Use your data to shape your choices between different ways of managing your community.What happened previously when different approaches were used? If you see that there are more product complaints than before, understand what they are and study the big picture trends. Make sure to circle back to your product team and work out a plan to operationalize the feedback.

What health indicators are you tracking in your communities? How are you measuring business value that they bring? What are some surprising things data told you that you didn't know before?

Title image courtesy of Bruno D'Andrea (Shutterstock)

Editor's Note: To read more by Maria check out Community Design is Like Throwing a Good Party - It Takes Planning