Feature
This winter, I was talking with a real estate agent friend. We had both started new jobs in 2021, and while our industries and day-to-day work is very different, it was fun to compare notes.
One topic that came up was how we measure whether we’re doing a good job or not. He told me his new firm was very data-driven and had scorecards for each agent at the firm.
My friend was particularly proud of a recent month where he had a conversion rate of over 100% and won an award at his new company! When I asked him how that was possible, he shared the calculation his firm uses:
# of houses sold divided by # of new houses listed for sale
I don’t know how many houses he actually sold, but for argument’s sake, let’s say he sold six houses in the month and had five new listings posted, for a conversion rate of 120%.
Bringing Context to Success
On the surface, a 120% conversion rate is excellent. My friend closed more houses than he even listed! But digging a little deeper had the potential to show that my friend was not in fact a star performer.
What if I told you that the average time to sell a house is 35 days, and the average time it took for my friend to sell those six houses was 92 days? Wouldn’t that paint a very different picture of how “successful” my friend had been during his “award-winning” month? This new information might suggest my friend is actually a low performer. Sorry pal!
I thought better than to start explaining to him how this was an example of a “Time Series” analysis and how “Cohort” was really a better measure of success. But I made a note of the conversation to ensure I’d remember to use it for this month’s article!
Related Article: The Challenges of Measuring Marketing ROI
Time for a Change in Marketing Measurement?
This real estate example is almost identical to how many companies measure success in the B2B software industry. Instead of houses sold, we talk about won deals, and instead of new listings, we talk about new opportunities, or going even further up-funnel, MQLs (Marketing Qualified Leads).
So let’s talk about how this plays out in the B2B software industry. For the purposes of this exercise, I’m going to talk about a theoretical B2B funnel.
At the beginning of each year and quarter, most companies set targets: Targets for MQLs, opportunities created, pipeline, annual recurring revenue and other KPIs. If the company hits the targets, it’s usually a good thing. If the company misses the targets, it’s usually bad. Right?
Not so fast. Remember last quarter when you crushed your MQL target? Well, if you are in B2B software, chances are those MQLs from last quarter will turn into pipeline this quarter. While your CMO and head of sales are congratulating you on creating a ton of pipeline this quarter and encouraging you to do more of what you are doing right now, the real hero is whatever you did last quarter. Go do more of that! What you are doing right now is not leading to your great quarterly performance; what you did last quarter is.
Related Article: Place Your Marketing Bets in 2022
Time Series vs. Cohort Series
Let’s break it down even more simply:
Time Series analysis provides a point-in-time view into what is happening, without any understanding of how past activities impact current and/or future performance.
A Time Series report would tell me how many MQLs you generated or how many opportunities you created in October. It tells you nothing of the quality of those MQLs or opportunities or what happened to them over time.
Cohort Series analysis looks at a particular window of time for an activity and tracks the performance of that activity over time.
A Cohort Series report for MQLs in October would tell you what happened to those October MQLs over the next X months. How many became opportunities and how many were won and lost.
Learning Opportunities
Webinar
May
30
ChatGPT and the Future of Conversational AI
This discussion explores various aspects of AI implementation, ethical considerations, and challenges for your organization.
Webinar
May
31
The Evolution of Employee Listening - 2023 Trends & Best Practices
Learn proven effective methods and solutions to supporting an inspirational future of Employee Voice
Webinar
Jun
1
How organizations can design, build and run Generative AI into the Customer Experience
Discover how to drive enhanced CX through Generative AI
Webinar
Jun
6
The Future of Social Media: Emerging Trends and How to Stay Ahead of the Curve
The impact of emerging technologies like AI on social media
Webinar
Jun
7
The Building Blocks of Personalization
This webinar will explore how personalization can create powerful ecommerce shopping experience.
Webinar
Jun
13
The Future of Contact Centers with AI at the Helm
Learn how to balance human intuition and interaction with the potential uses of AI to create delightful and lasting CX
Webinar
May
30
ChatGPT and the Future of Conversational AI
This discussion explores various aspects of AI implementation, ethical considerations, and challenges for your organization.
Webinar
May
31
The Evolution of Employee Listening - 2023 Trends & Best Practices
Learn proven effective methods and solutions to supporting an inspirational future of Employee Voice
A Time Series chart might look something like this:
TS Chart 1
| MQLs | Opportunities | Won Deals | Win Rate | |
| October | 1000 | 200 | 30 | 3% |
| November | 1000 | 300 | 50 | 5% |
| December | 1000 | 150 | 20 | 2% |
A Cohort Series chart might look something like this:
CS Chart 1
| MQLs | Opportunities | Won Deals | Win Rate | |
| October | 1000 | 300 | 50 | 5% |
| November | 1000 | 150 | 20 | 2% |
| December | 1000 | 200 | 30 | 3% |
If you measure your business based on Time Series, you’d think that November was your best month. You’d double down on all the programs and activities you did in November. Because, of course, you create the most opportunities in November and won the most deals in November! Your win rate was the highest in November, too.
But if you measure your business based on Cohort Series, you’d see that, in fact, October MQLs were the best and resulted in the most opportunities and won deals, and November was the worst.
You’d decide to double down on all the programs that resulted in MQLs in October and you’d look at your November programs as failures because November MQLs had the lowest win rate.
Related Article: Intent Data and the Gap Between Sales and Marketing
How to Shift From Time to Cohort
Convincing your leadership team to measure performance based on Cohorts instead of by Time Series can be extremely difficult. You’ll probably want to bang your head against the wall after every conversation, if we’re being honest. But ultimately, it’s the best way to measure success.
Here are some tips for how to convince others that Cohort is the way to go:
- Remind your leadership team that a B2B Sales Cycle can often be six-plus months. A new marketing campaign is like a new sales rep. There is a “ramping” period, and performance cannot be fully measured for at least six months. Results during the incubation period should really be attributed to the previous campaign.
- Show them the Cohort analysis. The chart I shared above is pretty convincing, isn’t it? I would be shocked if you don’t have similar cohorts in your business. You may have invested heavily in events in Q1 and in webinars in Q2, and the cohorts will likely look different. Data speaks.
- Everything we do in marketing is about ROI, right? ROI in anything (stock market, housing market, crypto market, gambling) is measured using Cohort Analysis, so why should marketing be any different? The results of the campaign you ran in October should be measured over six-plus months, just like the money you invested in Amazon stock. Your business results are just like a portfolio of activities. When deciding where to invest next, you should be looking at individual performance of campaigns (stocks) over a period of time instead of looking at aggregate performance of a single month.
Understanding the benefits of Cohort Analysis is not easy. It’s not easy to explain it either.
My last piece of advice is this:
Even if leadership doesn’t understand the difference or the value in Cohort Analysis, don’t let that stop you from using it to make the best decisions possible for your business. At the end of the day, you are getting paid to make good decisions, and using Cohort Analysis will result in good decisions for you and your business.
