As business leaders, we tend to rely on quantifiable metrics because — as the old adage goes — “numbers don’t lie.”
Except, of course, when they do.
Are Your KPIs Being Manipulated?
The topic of this article doesn’t quite align with the spirit of the holiday season. But it is critically important in customer experience (CX), especially during these times of the year when there is high visibility into what is happening in CX programs. Many of us have learned the hard way that you can’t always take the data coming from various teams at face value. Individual employees, teams and technology vendors often want to tell the best possible story about their programs, and some will have no problem manipulating the metrics to help them tell that story.
And when it comes to CX programs, it’s no different. In my most recent book, "Experience is Everything," my co-author Eng Tan and I discuss the KPIs that CX leaders need to track to ensure they’re best conveying their efforts to other company leaders. However, we also discuss how those same metrics can tell an inaccurate story in the wrong hands. Here are some real examples of how KPIs can be manipulated:
- CSAT: In order to maintain a good CSAT score, a customer service agent may grant an exception to a customer, like a full refund, when they’re not supposed to. The customer will be satisfied, but the company unexpectedly loses money.
- Average resolution time: Agents with bonus structures that incentivize speed over all else will accelerate escalation on inquiries that they know will be long in duration. They will find ways to avoid incoming inquiries on products that typically have long chat times, while rushing to take inquiries that they know can be resolved quickly and with high customer satisfaction (like forgotten passwords).
- First response time: Some organizations will only report first response times during normal business hours, as if your website was literally a physical storefront located in the eastern time zone. This paints an inaccurate portrait of how well the team is performing in terms of responding to customers. More importantly, it can give support organizations a pass for ignoring customers during off-hours.
- NPS: Many ticketing systems will automatically suppress the distribution of measurement tools like NPS surveys during certain circumstances (overnight, when certain macros and processes are being used, and so on). Customer service agents who have provided a poor experience to a customer will wait to resolve a ticket for that inquiry until those circumstances are met, meaning the customer never receives an NPS score and therefore can’t express their dissatisfaction.
- Deflection rate: Chatbot providers can artificially inflate the deflection rates they report to their customers. One common practice among chatbot providers is to proactively surface only the types of inquiries that it can handle to a consumer engaged in a chat. For example, a bot might limit the consumer’s options to avoid the number of unresolved interactions it escalates. A bot might ask, “What can I help you with today? Then offer just a few possible replies: “[Request a refund] [Where is my order?] [Edit my order].” Particularly in pre-sales situations, a consumer may try once or twice to find an answer to a question that exists outside the limited options offered by the chatbot. Then they’ll simply shop elsewhere. Meanwhile, the chatbot does not include this as a missed deflection. The inquiry is simply never registered at all.
Related Article: Inflation Is No Excuse to Cut Customer Experience Programs
Learning Opportunities
Webinar
Mar
21
Ready to Level Up Your Learning? Discover Your Learning Maturity Score
Learn how to level up your learning program in this free, interactive workshop
Webinar
Mar
29
The CX Commute vs. the CX Joy Ride
Is your CX traffic holding up your customer's journey?
Webinar
Mar
29
Is Your Microsoft 365 Data Recoverable? How to Survive the IT Hotseat When Data is Lost
See why MIcrosoft recommends third party backup
Webinar
Mar
30
An Optimized Customer Contact Strategy Combines Transparency and Intelligence: The State of Outbound Communications
Learn from the Neustar-commissioned Forrester Consulting study on challenges and solutions for contact centers
Conference
May
10
CMSWire CONNECT Customer Experience Conference - Austin 2023
Don't miss the top customer experience and digital experience conference of the year — live in Austin, Texas May 10-12.
Conference
May
10
Reworked CONNECT Employee Experience & Digital Workplace Conference Austin 2023
North America's best employee experience and digital workplace conference of the year — live in Austin May 10-12, 2023.
How to Protect Yourself Against Metrics Manipulators
Because of scenarios like these, we recommend implementing robust and stringent audit and quality/accuracy assurance protocols to ensure the numbers that CX teams report out reflect the real story of how customers are interacting with your brand. Many CX organizations only audit between 2% to 5% of their interactions. To truly protect yourself against metric manipulation, we recommend boosting that number significantly, to at least 15% of all interactions.
Another solution is to track a wider range of KPIs that are reported to gauge the success of your CX programs. As we showed you, many metrics that rely on customer surveys, like NPS, can be fairly easily manipulated. But what if you combined those with metrics around conversion rotates or customer lifetime value? Similarly, while the clever customer service agent might be able to game the system in their favor if they’re only being measured on efficiency, why not also hold them to certain interaction quality standards as well? The more KPIs you measure, the more they begin to act as a counterbalance against each other, preventing individual employees/teams from gaming one metric without adversely affecting the other.
Of course, the only way to 100% protect you and your organization from being caught in a KPIs lie is to instill a culture that truly values the customer above all else. When every single person involved in CX understands their important role in delivering an exceptional customer experience, and knows that they will be recognized for doing just that, then you will have an organization where the metrics are really impressive ... and reflect the actual experience of your customer.
That is what NOW CX is all about.
Learn how you can join our contributor community.