Editorial
The Gist
- Harm prevention. Systematic frameworks can identify potential harms pre-launch, saving costs.
- Usability focus. Usability testing with thinking style archetypes highlights unsupported features.
- Proactive consumer product safety measures. Combining harm categories and scale ensures broader user support.
Recently AirCanada was found liable for the inaccurate information that its AI chatbot provided which included a bereavement policy that did not exist. While the case did not cost AirCanada much in terms of money, it is significant. It set a precedent that if a company’s chatbot provides inaccurate information, the company is responsible.
Let's take a look at potential harms caused by products and how companies can better ensure consumer product safety and liability.
All too often all types of products, not just AI products, are launched into the marketplace with barely a glance at what could go wrong, who it could harm — and how to prevent the problems in the first place. In other words, consumer product safety is not getting enough attention.
The potential harm isn’t so much hidden as not looked for when it comes to consumer product safety. It doesn’t have to be this way. “It saves the organization money and reputation if you give the team permission, time, and budget to think through potential harms before launching a solution.” according to Indi Young, qualitative data researcher and author of "Time to Listen," "Practical Empathy" and "Mental Models."
Don’t Know Where to Start With Consumer Product Safety?
With so much at risk, it can seem overwhelming to know where to start. To begin, use frameworks need to have a more systematic approach for identifying and mitigating potential product harms and ensuring consumer product safety.
Categories of Harm List
In 2019, I created this Categories of Harm List in Table 1. This isn’t the only list of this type nor is it exhaustive, but includes common areas of concern and can be adapted to include product or industry considerations. You can start with the nine categories as listed, add categories that your organization commonly gets negative feedback about and subtract categories that don’t apply.
Examples of Real Product Harms
Here are some examples of harms based on real products. You will see some examples that can fall under more than one harm category.
Financial:
An online investment platform showed customers inaccurate negative balances. Novice investors thought they owed the company large sums of money from options trading.
- Distressingly, this also belongs in the health harm category because the extreme emotional stress led to at least one suicide.
Fairness/Equity:
People on an online neighborhood app were reporting situations as suspicious simply because the people involved were people of color, making people of color feel targeted and unwelcome whether they lived in the neighborhood or were visiting.
- This example also qualifies as a safety harm because the people who were reported could be subject to harassment or police scrutiny for unfounded suspicions. The app has made changes to its algorithm and provided education to users on what to report to reduce incidences of racial profiling.
Safety:
A fitness app that encouraged users to compete with each other had a strong user base of military personnel. The app had a feature that inadvertently revealed to adversaries the location of military bases and other intelligence operations.
- This example extends into privacy harm as well, since data from the app could potentially be used to identify specific military personnel or their residences.
Control:
Despite public outcry, a communications app initially rolled out a new feature that allowed anyone outside of a particular channel to email people without an opt out option. For example, someone in a work channel could receive messages from a persistent salesperson with no option to block them.
- This email option also could become a health harm because it initially would have allowed stalkers and other bad actors messaging access without any filtering ability. The company quickly changed this feature to opt in after launch due to the criticism.
Consider the Scale of Harm & Consumer Product Safety
Zeroing in on some Categories of Harm and imagining scenarios is not enough to protect users and ensure consumer product safety. You need solid measurements and detailed data to persuade leadership that a harm needs to be addressed.
Indi Young explains that measurements stem from recruiting participants for a usability test who embody the “thinking style” archetypes identified by the organization's previous research. These archetypes are demographic-neutral profiles that describe individuals based on their goals rather than solutions. Usability tests often reveal that certain thinking styles are not adequately supported by the feature being tested. The researcher marks the extent to which the feature fails that thinking style using a “Scale of Harm” shown in Table 2.
Webinar
Sep
25
CMS Briefing: A Live Look at What’s Next in AI-Driven Platforms
Learn how leading organizations are using AI‑driven tools to publish faster, personalize smarter and stay secure.
Register
Webinar
Sep
25
Call Spoofing Trends Financial Institutions Can't Afford to Ignore
Don't let robocalls sabotage your customer relationships. Learn how to secure your voice channel.
Register
Webinar
On demand
Ready or Not: How Data-First Organizations Are Unlocking Agentforce Potential
Learn how to cut through the noise, activate Agentforce and build a Salesforce AI strategy that actually delivers.
Watch Now
Webinar
On demand
CX Reality Check: What Your Customers Are Really Thinking
Register now and get the insights you need to not just meet rising expectations, but exceed them.
Watch Now
Webinar
On demand
Content Leaders Collective: What a Good CCMS Actually Looks Like
Discover how content leaders transform clunky systems into growth engines — and why your competitors are ahead.
Watch Now
Webinar
On demand
From Hype to High-Impact CX Strategies That Actually Scale
Turn buzzworthy AI and outsourcing trends into measurable CX wins with fresh 2025 data.
Watch Now
Young states, “Most organizations search for and take care of Mild Harms, but are not proactively looking for Serious, Lasting, or Systemic Harms.”
Instead of waiting for negative news stories or product safety lawsuits, your team can proactively identify these more severe levels of harm like the examples here:
Combine Categories & Scale of Harm
To persuade leadership that it’s valuable to take time to avoid potential harms and consumer product safety, it’s powerful to combine the Categories of Harm with the Scale of Harm measurements. In a usability test, for each feature that fails a thinking style, researchers can note both the Category and Scale of Harm.
Identifying the potential Categories of Harm and measuring the Scale of Harm early in the process provides the opportunity to eliminate or mitigate it before a solution is launched. The organization has a chance to make solutions to support a broader range of people. This investment on the front end helps avoid financial and reputational pitfalls.
Learn how you can join our contributor community.
