Say What Why Sentiment Analysis Continues to Fail Digital Marketers

Technology can help filter the mighty river of unstructured data flowing into your company’s marketing department, especially from sources like social media. But does it really enable you to understand what your customers are saying?

Does a tweet about a "cool pool" mean a hotel offers ideal water-based recreation -- or simply keeps the water too cold? Does a Facebook post that claims "you have to see it to believe it" reflect sincerity or sarcasm?

It's not always easy to find out, according to Alison Smith, an analyst at Forrester Research who specializes in sentiment analysis.

Though networks like Facebook and Twitter offer goldmines of insight about what customers say and do, the bigger challenge is looking for the meaning behind the words. That means digging deep enough to uncover emotions, opinions and attitudes -- a potentially far more challenging task.

Fatal Errors

“If we’re talking about exterminating bugs, works like ‘killing,’ ‘dying’ and ‘death’ would be positive,” Smith told CMSWire. “But if we’re talking about pharmaceutical drugs, ‘killing,’ ‘dying,’ and ‘death’ would be extremely negative.”

Understanding the contextual implications of words is no easy task for man or machine. Smith cited one recent academic study that showed humans reading the same passage could agree on its meaning only about 80 percent of the time.

“So there’s sort of a disconnect between users thinking that their sentiment tool should get them 95 percent, 98 percent or 100 percent accuracy when you think about how humans will only agree a certain amount of time,” she said.

Take the term “sentiment analysis.” What does that really mean? According to Smith, it is the technology of making the large amount of data digestible. The problem is that most sentiment analysis tools can’t discern the context because they rely on general algorithms.

So, Smith jokes, it can be hard to know what someone means when they say “this vacuum really sucks.” Will we ever know the answer to that question with 100 percent certainty? “I don’t think we’re every going to get there,” said Smith.

The Search for Meaning

While Smith noted the imperfection of sentiment analysis, others focus on its gains. Take Erin Olivo, a clinical psychologist, assistant clinical professor of medical psychology at Columbia University and an advisor at SmogFarm, a company that measures, tracks, and aggregates millions of individual emotions in real time.

Olivo, speaking at the Sentiment Analysis Symposium in New York City last year, said, "There is more and more good information coming all the time" with regard to the analysis of data. She was one of 18 speakers at the day-long conference, who shared their insights on maximizing the business value from the opinions, emotions and attitudes expressed in social media, news, and enterprise feedback.

Companies like SmogFarm are moving the dial on sentiment analysis beyond simple positive and negative emotions. Through the use of a custom built algorithm that is able to track “emotional drivers” — keywords and phrases that are linked to specific emotions — SmogFarm is providing insight on feelings including anger, disgust, excitement, fear, happiness, love, sadness and shame.

The Sentiment Analysis Symposium is organized and produced by Alta Plana Corp., which was founded by Seth Grimes in 1997 to deliver business analytics strategy consulting and implementation services with a focus on advanced analytics. Grimes is a big believer in the meaning behind the words. In a post on his blog, Breakthrough Analysis, he writes:

Sentiment, mood, opinion, and emotion play a central role in social and online media, enterprise feedback, and the range of consumer, business, and public data sources. Together with connection, expressed as influence and advocacy in and across social and business networks, they capture immense business value."

The next Sentiment Analysis Symposium will be held March 5 and 6 at the New York Academy of Sciences in Lower Manhattan.

Mostly Right

SmogFarm is not unique. A host of companies including Crimson Hexagon, Salesforce Radian6, Netbase and Converseon have all developed sophisticated products and services around sentiment analysis.

How accurate are the results? As any investor or gambler can tell you, there is enormous value in being right even 80 percent of the time. Even though it is never 100 percent accurate, sentiment analysis still provides reliable insight into how the public is reacting to a new product, a recall, flight delays, or other events affecting large numbers of customers.

“If we were trying to rely only on human analysts to go through the volume of tweets that a brand like McDonald’s gets – they’re measuring tweets per second – it’s impossible,” said Smith.

Even skeptics like Smith thinks sentiment analysis will live on. “While it’s not as good as perhaps it should be, or as the expectation is, it is much better than nothing,” she said.

Title image by Login (Shutterstock).