For too many, improving customer experience means relying on an occasional static survey instrument or quickly analyzing customer data in the context of an emergency. But designing a successful customer experience (CX) program requires constant iteration and delivery — and should be a seasonal process that frequently tests new initiatives and drives decision-making.
This seasonal process should consist of seven intricately-linked steps. So far in this series, we’ve talked about the first two: customer experience program design (creating an overarching, "big picture" plan to improve customer experience over time) and customer experience project design (taking the general, big picture principles and goals of your program and applying them to specific actions).
The third step in this process is sample design.
What Is Sample Design?
Sample design is the framework you use to involve the right customers in your CX program and projects. Each time you design a sample for any project, you lay an indispensable foundation for creating reliable and functional insights for your organization.
If you can readily answer questions like, “Which customers should I select for my project?” “What is the addressable market for my product or service?” and “What criteria did I apply?” — then you are ahead of the game in CX. You should know how many customers you have, to whom you are marketing and a clear idea of natural customer segments. If you have the numbers “somewhere” or need time to pull them together, it is time to prioritize broad internal socialization of these metrics.
Most executives and other stakeholders will care that you have selected the right group of customers. Before you author a study, make sure you know who your sample will be. The kind of study you are undertaking and the actions you plan to take should inform your sampling methodology.
7 Considerations for Sample Design
1. Randomness Is Everything
Whether you are a biologist or social scientist, the way you select your sample is what makes your research replicable. The only way to get close enough to confidently say, “This is the case!” is by ensuring your sample is “really random.” Unintended criteria or unexamined periodicity in samples can impact analysis and lead to assigning meaning where there is none or missing an influencing condition. (Note that other probability-based methods or non-probability sampling methods can also be very useful in advancing a CX program. It is important you understand which metrics your stakeholders are looking for).
It’s more important to drive CX design improvements than argue internally about statistical theory. Vetting your sample for randomness or using clear criteria for pulling sample will answer any questions quickly, putting you back on track to respond to customer needs. Be able to quickly show the selection criteria and confirm that the sample was selected entirely at random from that group.
2. Plan For and Avoid Sample Error
Assuring randomness is more difficult than it appears. Some customers are more likely to answer online. Some never answer. Most only reply when they need a response. Because this is your reality, you need to have a clearly-defined and socialized sample plan — before you field a CX project. Consider the sampling plan for any project individually. Good CX programs work this into project design motions because there is no such thing as “program sample.”
Related Article: Not Another @#$&! Survey ...
3. Use Temporal Criteria
Let’s say you want to survey every customer who called your Dallas call center from January to March. You want to know which calls do or do not have contact information, and you want to look at the percentage of customers who had their problem resolved in one call, as well as have a plan to address those who did not. Select your sample carefully to match the goals of every study, especially in a transactional study.
Learning Opportunities
Webinar
Mar
21
Ready to Level Up Your Learning? Discover Your Learning Maturity Score
Learn how to level up your learning program in this free, interactive workshop
Webinar
Mar
29
The CX Commute vs. the CX Joy Ride
Is your CX traffic holding up your customer's journey?
Webinar
Mar
29
Is Your Microsoft 365 Data Recoverable? How to Survive the IT Hotseat When Data is Lost
See why MIcrosoft recommends third party backup
Webinar
Mar
30
An Optimized Customer Contact Strategy Combines Transparency and Intelligence: The State of Outbound Communications
Learn from the Neustar-commissioned Forrester Consulting study on challenges and solutions for contact centers
Conference
May
10
CMSWire CONNECT Customer Experience Conference - Austin 2023
Don't miss the top customer experience and digital experience conference of the year — live in Austin, Texas May 10-12.
Conference
May
10
Reworked CONNECT Employee Experience & Digital Workplace Conference Austin 2023
North America's best employee experience and digital workplace conference of the year — live in Austin May 10-12, 2023.
4. Don’t Oversample
If selected randomly from any population, 384 people is a sufficient sample. Your results will be within a 10% range 95% of the time. A higher level of confidence becomes much more expensive at this point. Remember that you can sample 500 respondents selected randomly from a population, and they will closely reflect the actual population 95% of the time. This bears repeating: the secret to success is random selection. Don’t overwork sampling design. A sample methodology should match the population under study.
5. Confront Low Sample Sizes and Response Rates
If you try to make one sample the source for many abstract questions, your program will suffer. If you focus on the right sample, you can ask a targeted question at the right time. Your response rates and completion rates will soar as your customers begin to see that communication with you is unobtrusive, easy and results in a benefit for them.
Related Article: 5 Ways to Optimize Your Surveys for Better Customer Experience
6. The Mode Matters
Match your survey method to the sample you select (and vice versa). Who are your current customers? Do they have different characteristics or preferences from your potential market? If you sample grandparents, don't expect responses from a QR code. A mixed method approach can help drive action across different customer segments.
7. Action Is the Goal
Too many CX programs work in perpetuity, ignoring inconvenient feedback. If I let you know about a particularly bad experience I had with your company, an incoming phone call from the CEO or the local manager may gain your company a customer for life. Default to actual numbers and people whenever possible. At some point, everyone needs to be reminded that the goal of CX is to increase the behaviors that drive revenue by creating great experiences.
A great customer experience often means the customer was treated like a person rather than a number. Use a well-selected sample that follows the assumptions of your quantitative method to make these customer interactions real. Carefully selecting your sample will prepare you well for questions from your executives and stakeholders and, more importantly, drive insights into actions for individuals.
Editor's Note: This is the third article in a seven-part series on customer experience design. Check back soon for the next installment.
Learn how you can join our contributor community.