Measuring Business Communities Are We Doing It Wrong

It's unfortunately still the case that most companies don't understand community, judging by job postings on job boards and attempts to measure its ROI.

But it’s not their fault. Because community is not a discreet set of activities but rather, a foundational communication platform. Calculating the ROI of community is a bit like calculating ROI of email – as difficult as it is pointless. And yet understanding the business impact of communities is critical to being taken seriously by company executives.

Most of the challenges in defining -- let alone measuring --, communities stem from the following:

The Issues

Lack of definition

If you can’t describe something, how can you measure it? Such is the problem with communities in many organizations -- and an overall problem with our discipline. There isn’t much clarity around what community management is, what community managers do and where in the organization community “sits.”

The relative organizational success of other job functions, such as marketing or sales, is inextricably tied to clarity in departmental boundaries, job descriptions, organizational dependencies, reporting structures and budgets.

Let’s take marketing. When someone says “marketing,” you with some certainty can understand what is being described. While just about every business discipline is experiencing massive change, there’s usually a baseline agreement. I hate definitions as much as the next gal, but words matter, and lack of clarity makes it hard to do the job well.

A new model

This lack of agreed-upon definition is caused by the inherently cross-functional nature of community management. It’s not the kind of cross-functionality where you simply keep other teams updated – rather, it provides the connective tissue for the entire business, inside as well as outside. A community is a platform for activities to take place and for people to connect and work together, not a set of discreet activities themselves. And when you can be anything to anyone, you risk becoming nothing.

If it’s something that we can’t define, it means that we can’t build a strategy and hire the talent to execute on that strategy. The conflation between “social media” and “community” is not helping us either. And if we don’t know what something is, how can we measure it?

Not strategic enough

And herein lies the dilemma. Because community is everywhere and can be so many things to so many people and departments, no one knows where to put it. Is it marketing? Is it support? Is it customer success? Yes and yes and yes. To make matters more complicated, a well-run community “just happens” to the point where the how becomes invisible and the what becomes a given. While it’s many community manager’s ultimate goal to create a self-sustaining community, doing so may put it “out of sight, out of mind,” resulting in neglect.

Ultimately, community isn’t recognized as its own business function and ends up reporting to, most often, marketing or support. While the intention may be there to lead a cross-functional team, when push comes to shove, you are responsible for the unit where you report – and what gets measured, gets done.

To be a truly cross-functional foundational platform, community should be its own department that has its own operating mandate and is tightly coupled with other parts of the organization. By being someone’s line item with funding one year and without funding the next, it will never be strategic enough.

What Do We Measure?

None of these issues will get resolved any time soon. It is my hope -- and a personal goal of mine -- to see communities elevated to the necessary strategic level inside of their companies. For that to happen, we as community managers have a lot of work ahead of us.

To be fair, I don’t think we (collectively) have done a compelling enough job of portraying community as a business necessity and not an add-on, whose budget may or may not get slashed. Demonstrating impact has never been more important in order for us to be taken seriously. I put community metrics and KPIs into two main buckets:

Direct impact

Community metrics: These are the traditional measurement of community metrics, which usually fall in the general buckets of adoption and engagement.

  • Roughly speaking, adoption stands for how people are adopting it -- do they know about it, are they logging in, are they using it in their flow of business and hopefully replacing ineffective processes with it. Adoption metrics focus on how effective you are at changing behaviors and gaining mindshare.
  • Engagement metrics measure how people interact with community and how they contribute back to it. Engagement metrics span across: creating (original posts, documents and other artifacts), sharing (existing content), commenting (on existing content), acknowledging (“liking” and such).

Community metrics are incredibly important to understanding the ongoing health of a community. Using absolute numbers or benchmarking against other companies is a fool’s errand -- what does “30 percent original posters” actually mean? What’s a lot more telling is tracking these as a trend, where the trajectory can be a lot more telling than any absolute number. Tracking metrics over time can point to a sudden shift and help you understand trends.

Business KPIs: While measuring community metrics is important -- and should be done with meticulous understanding and precision -- engagement metrics are hardly compelling for leadership. They want to understand the impact that community is having on business KPIs.

Go back to the initial use cases that you planned on supporting with community. Was one of the goals to deflect support calls? Measure reduction in call volume. Was it speed of response? Measure that. Was it customer loyalty? Measure repeat purchases, order size, time between purchases and NPS.

Indirect impact

Mapping cause to effect directly is difficult. There are many confounding factors. But because community is truly a new model of communicating, its effects are going to spill out onto other adjacent processes. While your original goal may have been around reducing support time, it may indirectly aid building your advocacy program and thus aid marketing objectives. Look for these “spillover” effects, and document them well. The more you can convince people that community is helping departments all over the company, the more likely you are to be seen as a must-have foundational business practice, not just an add-on.

Understanding and measuring community is never going to be as clean-cut as other disciplines, but since the future is networked, community stands to become the hero of the organization. Be strategic about things you promise and deliver, and don’t get too stuck in thinking about engagement and adoption – always ask “now what?” and “how does this drive the business forward.”

Measurement difficulties abound, especially since attribution is hazy at best, and the org chart shuffle is still to be decided. Investing in community for the long term is the only way to go, since its impact on the business almost always take time.

Title image by Elliot Gilfix (Flickr) via a CC BY-NC-ND 2.0 license