robot banana graffiti

Lessons Learned from a Chatbot Failure

6 minute read
Dom Nicastro avatar
What can we learn from chatbot failures? As it turns out, a lot.

Have you ever had a chatbot set up a meeting with someone you've never heard of via calendar invite, respond with “bye” after you decline the invite, then prompt a human to enter the email chain who calls you by the wrong name?

I have. 

We get a lot of emails from vendors, but they usually involve press pitches. This was a first. A vendor, through its chatbot, set up an unprompted meeting with me. Things got stranger when I engaged. While the exchange was surreal and entertaining in equal parts, it was a good example of where bot technology and artificial intelligence can fail and, even more importantly, a reminder for brands to do better deploying such tech into their sales and marketing processes.

“A common mistake today is that companies still view chatbots as an alternative to human-to-human communication channels when the reality is companies need both,” Morgan Molnar, senior product marketing manager at SurveyMonkey, told CMSWire. “It’s no longer a matter of ‘bots vs. humans.’ Instead, the goal of modern businesses should be to make it as easy as possible for people to start conversations and buy on their terms, in real-time, when it’s most convenient for them.”

Chatbot Says 'Bye'

Back to my bot story. Here’s the chronology of the exchange. For all I knew this was a human setting up a meeting with me, until an "air of bot-ness" fell over the interaction. Here’s what happened after I declined a random calendar invite from an unknown person the morning of July 15:

July 15: 11:20 a.m.

email screenshot with a bot saying, "bye"

July 15, 11:40 a.m.

Email exchange between a bot and CMSWire reporter Dom Nicastro

July 15, 12:04 p.m.

Email exchange between sales rep and CMSWire reporter Dom Nicastro

July 15, 12:24 p.m.

email exchange between CMSWire reporter and a sales rep

Related Article: A Good Chatbot Is Hard to Find

What We Can Learn from Chatbot Failure

Clearly this wasn't the first time a sales or marketing technology failed a reputable brand. The company's chatbot let the sales rep down as much as it added a touch of the surreal to my Monday morning. Even when the sales rep was prompted to chime in based on the exchange between me and the bot, the technology wasn't good enough to let her know I was Dom and not Adrian.

Technology fails us. Even the biggest brands have had their share of AI gone wrong. But these failures also teach us. Marketing and sales are all about experimentation. Is this the right message? Is this the right lead? Learning from the tech failings along the way to deliver better on future marketing and sales campaigns is an intrinsic part of this culture of experimentation.

Related Article: Don't Let Chatbots Drag Down Your Customer Experience

Learning Opportunities

Knowing Your Customer's Pain

SurveyMonkey’s Molnar believes these type of problems stem from understanding where customers and prospects have frustrations. What common mistakes do entities make trying to deploy a chatbot? What can they do to get better? “They can make the experience better by recognizing the top frustrations people are facing online today,” Molnar said. She cited the 2019 State of Conversational Marketing report by Drift and SurveyMonkey that found these customer frustrations:

  • Getting answers to simple questions.
  • Dealing with websites that are difficult to navigate.
  • Finding basic details about a business (e.g. address, hours of operation, or phone number).

“Taken together, these frustrations point to a broader underlying issue that’s been plaguing online experiences for years: People can’t find the information they’re looking for quickly and easily,” Molnar said.

Chatbot Frustrations Persist

According to a survey by Helpshift, 50.7% of respondents cited being kept away from a live person as the top challenge of using a chatbot. Next was too many unhelpful responses (47.5%), redirects to self-serve FAQs (39.5%), bad suggestions (28.2%), pop-up chatbot prompts (25.0%), unnecessary pleasantries (24.9%) and takes too long to respond (24.2%). Further, the Qualtrics XM Institute in its State of Customer Experience, 2019 found that when companies evaluate the customer experience they deliver acrosschannels, they rated "on the phone with an agent" and "in a store/branch" the highestand "via chatbots" and "across multiple channels" the lowest.

There was no answer choice reserved for, “Bot sent me a calendar invite, said ‘bye’ when I declined, kicked in a human sales rep, who then thought I was a woman named Adrian.” The point? There's clearly work to be done in chatbot land.

Related Article: Keep These 3 Things in Mind When Introducing Chatbots to Customer Service

Chatbot Potential Is There

Chatbots can provide value, convenience and a customer-centric approach, serving as a supplement to human customer service representatives, according to Molnar. But automation is not synonymous with “set it and forget it.” The same care and attention to detail needs to be applied to chatbots which applied to the brochure mailers of the past.

“The integration of AI in marketing will have numerous benefits, all of which will streamline marketing processes, and ensure improved return on investment of marketing efforts,” said Katherine Rowland, digital marketing executive for Your Parking Space. “Marketers can use AI to identify potential clients or buyers and deliver the ideal content that’s most relevant to them. Chatbots will be able to automate chat functions and other direct-to-consumer engagement avenues, saving time and expenditure.”

Tips for Chatbot Success

Chris Karnes, head of growth for Going Merry, said he has gone through a product development and product launch for an AI chatbot and is “all too familiar with the fears/misconceptions that come with AI.”

He shared these tips for consideration when deploying chatbots: 

  • Ensure your audience knows it's an AI-driven solution. Don’t try to trick them. They'll always know and that destroys trust. “Transparency around a technology-driven will immediate create a bit more patience around the experience,” Karnes said. “Users are used to dealing with technology and know it's not perfect; they expect more if they think it's a human.”
  • AI should be a user-first experience. Focusing on how the user wants to work ahead of what the business wants will provide more data with more actionable value.
  • Test it manually before you build it. Some AI solutions seem great on paper, but just don't work for users in real-life. Prototype a basic solution and get user input, fake the AI for your first few interactions to learn how users will break it. "There's nothing worse," Karnes said, "than AI that isn't actually intelligent."

About CMSWire

For nearly two decades CMSWire, produced by Simpler Media Group, has been the world's leading community of customer experience professionals.


Today the CMSWire community consists of over 5 million influential customer experience, digital experience and customer service leaders, the majority of whom are based in North America and employed by medium to large organizations. Our sister community, Reworked gathers the world's leading employee experience and digital workplace professionals.

Join the Community

Get the CMSWire Mobile App

Download App Store
Download google play